File Journalizer

Welcome to the File Journalizer project !

This project provides a command-line utility for tracking changed files and directories over time in a history-/journal-file.

A journal file contains the file path, file size, last modified time, scan timestamp and checksum of the scanned files.
The utility can be used for scanning, searching, comparing, listing the change history and even finding duplicate files.

Download...
FileSize  Date
20240820/file-journalizer-1.0.10.zip 161 kB  2024-08-20 

Installation...
Prerequisites:
This application requires a Java Runtime Environment (JRE). For Example: Java Platform, Standard Edition

Quick Installation:
1. Download the distribution archive (.zip) file using the link in the "Download" section.
2. Extract (unzip) the downloaded archive to a new directory.
It contains a shell-script bfj.sh (for Linux) and a batch-file bfj.bat (for Windows).
The utility can also be invoked with the command: java -jar bfj.jar

Examples...
bfj.sh help
bfj.sh scan "/home/user" "home-user.bfj"
bfj.sh dump "home-user.bfj"

Command Syntax...
Syntax: [action] {options} [parameters] {options}

--------
Actions:
--------
help                  Display this help information
scan                  Scan files and update journal file
dump                  Dump journal entries to console
list                  List journal entry paths to console
size                  Compute total file size of latest journal entries
test                  Test integrity of journal file
export                Export journal entries to another journal file
compare               Compare journal entries with another journal file
duplicates            Find duplicate files using checksum
reconstruct           Generate commands to create directories and symlinks

---------------
Common options:
---------------
--help                Display this help information
--version             Display the version information
--verbose             Verbose output
--debug               Debug output (also verbose)
--journal [file]      Incorporate entries of another journal file (read-only)
--sequential          Use sequential order of journal files (ignore scan timestamp)
--ignore-case         Case-insensitive subsequent --include/--exclude paths (ASCII)
--unicode-case        Case-insensitive subsequent --include/--exclude paths (Unicode)
--match-case          Case-sensitive subsequent --include/--exclude paths
--include [regex]     Include path (regular expression)
--exclude [regex]     Exclude path (regular expression)
--replace [x] [y]     Replace path (regular expression)
--arguments [file]    Read command arguments from a text file (line-separated)

--------------
Action "scan":
--------------

Scan files and update journal file.

Syntax: scan [scanPath] [journalFilePath]

Note: The common options --include and --exclude can be used for filtering paths.

Scan Options:
--dry-run             Simulation mode (without journal update)
--skip [regex]        Skip path from scanning (regular expression)
--size-only           Compare file size only (ignore last modified) 
--sum-only            Compare checksum only (ignore last modified)
--checksum            Compare checksum of files (always)
--follow-symlinks     Follow symbolic links (recursive)
--ignore-missing      Ignore missing/deleted paths
--ignore-errors       Ignore errors and continue
--java14              Use Java 1.4 implementation (java.io)
--java17              Use Java 1.7 implementation (java.nio)
--uncompressed        Append new entries without compressing
--summary             Display the summary after scan completion

Examples:
scan "/home/user/files/" "/home/user/files.bfj"
scan --dry-run --summary "/home/user/files/" "/home/user/files.bfj"
scan "/home/user/files/" "/home/user/files.bfj" --journal "/home/user/history.bfj"
scan "/" "root.bfj" --ignore-errors --verbose --skip "/dev/.*" --skip "/var/log/.*"
scan "C:\" "c-drive.bfj" --skip "C:\\Windows\\servicing.*" --ignore-errors

--------------
Action "dump":
--------------

Dump journal entries to the console.

Syntax: dump [journalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

Dump Options:
--timestamp           Include scan timestamp in the output
--checksum            Include file checksum in the output
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")

Examples:
dump "/home/user/files.bfj"
dump --timestamp "/home/user/files.bfj"
dump --checksum --timestamp "/home/user/files.bfj"
dump --checksum --timestamp --until 20170829123057 "/home/user/files.bfj"
dump "/home/user/files.bfj" --checksum --timestamp --since 20170829000000 --until 20170829123057

--------------
Action "list":
--------------

List journal entry paths to console.

Syntax: list [journalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

List Options:
--path-only           Display path of entries only
--ascending           Sort paths in ascending order
--descending          Sort paths in descending order
--include-missing     Include missing/deleted paths
--missing-only        Include missing/deleted paths only
--ignore-files        Ignore file entries
--ignore-dirs         Ignore directory entries
--ignore-links        Ignore symlink entries
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")

Examples:
list "/home/user/files.bfj"
list "/home/user/files.bfj" --ascending --ignore-dirs --ignore-links --path-only
list "/home/user/files.bfj" --ignore-case --include ".*filter.*" --include-missing
list "/home/user/files.bfj" --missing-only --ignore-dirs --ignore-links --ascending
list --since 20170829000000 "/home/user/files.bfj"

--------------
Action "size":
--------------

Compute total file size of latest journal entries.

Syntax: size [journalFilePath]

Note: The common options --include and --exclude can be used for filtering entries. 

Size Options:
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")

Examples:
size "/home/user/files.bfj"
size "/home/user/files.bfj" --ignore-case --exclude ".*\.zip"
size "/home/user/files.bfj" --include "/home/user/files/filter.*"

--------------
Action "test":
--------------

Test integrity of journal file.

Syntax: test [journalFilePath]

Test Options:
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")

Examples:
test "/home/user/files.bfj"
test "/home/user/files.bfj" --until 20170829123057

----------------
Action "export":
----------------

Export journal entries to another target journal file.
Multiple source journal files specified with the --journal option will be merged.

Syntax: export [journalFilePath] [targetJournalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

Export Options:
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")
--uncompressed        Export entries without compressing

Examples:
export "/home/user/files.bfj" "/home/user/export.bfj"
export "/home/user/files.bfj" "/home/user/export.bfj" --uncompressed

-----------------
Action "compare":
-----------------

Compare journal entries with another journal file.

Syntax: compare [journalFilePath] [anotherJournalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

Compare Options:
--size-only           Compare file size only (ignore last modified) 
--sum-only            Compare checksum only (ignore last modified)
--checksum            Compare checksum of files 
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")
--delta [file]        Write delta to a journal file      
--dry-run             Simulation mode (without delta journal update)

Examples:
compare "/home/user/files.bfj" "/home/user/other.bfj"
compare "/home/user/files.bfj" "/home/user/other.bfj" --checksum
compare "/home/user/files.bfj" "/home/user/other.bfj" --delta "/home/user/delta.bfj"
compare "/home/user/files.bfj" "/home/user/other.bfj" --replace \\ /
compare "/home/user/files.bfj" "/home/user/other.bfj" --replace "(C:\\)(.*)" "/c-drive/$2"

--------------------
Action "duplicates":
--------------------

Find duplicate entries using checksum.
Optionally, generate commands to remove duplicate files for a pattern. 

Syntax: duplicates [journalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

Duplicates Options:
--remove [regex]      Generate commands to remove duplicate files (regular expression)
--windows             Generate commands for windows: "erase"
--unix                Generate commands for unix: "rm"

Examples:
duplicates "/home/user/files.bfj"
duplicates "/home/user/files.bfj" --ignore-case --include ".*\.zip"
duplicates "/home/user/files.bfj" --include ".*\.zip" --unix --remove "/home/user/dir/.*"

---------------------
Action "reconstruct":
---------------------

Generate commands to create directories and symlinks.

Syntax: reconstruct [journalFilePath]

Note: The common options --include and --exclude can be used for filtering entries.

Reconstruct Options:
--windows             Generate commands for windows: "md"
--unix                Generate commands for unix: "mkdir", "ln -s"
--ignore-links        Ignore symlink entries
--include-missing     Include missing/deleted paths
--missing-only        Include missing/deleted paths only
--since               Since scan timestamp (Format: "yyyyMMddHHmmss")
--until               Until scan timestamp (Format: "yyyyMMddHHmmss")

Examples:
reconstruct "/home/user/files.bfj"
reconstruct "/home/user/files.bfj" --include ".*filter.*" --ignore-links --missing-only
reconstruct "/home/user/files.bfj" --windows --replace C:\\ F:\\test\\
reconstruct "/home/user/files.bfj" --windows --replace "(C:\\)(.*)" "F:\\test\\$2"
reconstruct "/home/user/files.bfj" --unix --replace "(/)(.*)" "/test/$2"
reconstruct "/home/user/files.bfj" --unix --replace "(.:\\)(.*)" "/test/$2" --replace "\\" "/"