The Data Carver Processors are a combination of Perl scripts and other programs that are designed to break up the recovered files into manageable chunks. As the script runs over the files, it will create a series of web pages with thumbnails and a second web page for each file that contains plug-in output like metadata, hashes, and etc. The scripts, for the most part, will not process damaged files. If a file is damaged, there will be no image for it on the web page. Just some text on the file.

Download

License: GPL 2.0
Download at: data_carver_processors.tar.gz (All of the data carver processor scripts are included in this file)

Details on the processors and files in the tar file is below. All of the data_carver_processors.tar.gz are the same file. It includes all of the data carver processors.

INI File

The INI file (data_processor.ini) contains the user configurable options for each one of the data processor scripts.

Each line has a comment before the parameter. See the INI file for more details.

Other Notes

Feedback: Please send me an email, cs[at]citadelsystems.net, with any features/plug-ins you would like to see. If you find any errors with the scripts, let me know. I am also interested any plug-ins you want to share.

Errors: As the script runs over the files you may see some errors outputted. The errors are from the programs running on the recovered files. Not all of the files that the data carvers recover are good files. Hence, the errors.

Image Processor

Point this script at a directory full of images and it does:

• Creates a series of web pages that contains a thumbnail of all readable images

• Gathers details about the files such as Exif data

• Sort images based upon nudity; Creates a CSV file with results in addition to the output in the web pages; Currently only detects "white" skin

• Review images using StegDetect (only does JPEGs); Creates a CSV file with results in addition to the output in the web pages

• Create your own plug-ins

Requirements
Perl modules: Getopt::Long, Pod::Usage, Image::ExifTool, Image::Magick, Imager::SkinDetector, File::Basename, Config::IniFiles
Libraries and packages installed: Imagemagick and Stegdetect
Stegdetect Notes: In order to get Stegdetect to compile using gcc 4.x, download the patch at Stegdetect 0.6.

Example commandline:
1) cd into the directory where you want the output put into
2) /appl/data_carver_processors/image-processor.pl --inputdir /foremost/output/jpeg --output index --plugindir /appl/data_carver_processors/image-plugins  --ini /appl/data_carver_processors/data_processor.ini --nudity --stego

ample Main Page

Sample Secondary Page
(Where you clicked on the image)

INI File

The INI file (data_processor.ini) contains the user configurable options for each one of the data processor scripts.

Each line has a comment before the parameter. See the INI file for more details.

Plug-ins

The script will only except plug-ins written in Perl. Just place your plug-in into the appropriate directory (i.e. image-plug-ins). Each plug-in has to be able to accept one command line argument which is -i {file name}. The Image Processor script will execute the plug-in and pass the -i option and file name to the plug-in. The output will be captured and placed into the web page for that file.

An example Perl plug-in

#!/usr/bin/perl 
#use warnings;
use Getopt::Std;
use File::Basename;
use Digest::SHA qw(sha512);
sub usage() {
 print new;
$sha->addfile($file);
$digest = $sha->hexdigest;
$digest =~ tr/[a-z]/[A-Z]/;
print "SHA512: $digest\n";

An example Perl plug-in with shell commands

#!/usr/bin/perl 
#use warnings;
use Getopt::Std;
use File::Basename;
sub usage() {
 print <
 usage: $0 [options] [directory]
 options:
 -i file    Input file
EOM
 exit;
}
usage() unless getopts("hi:");
usage() if $opt_h;
$file = $opt_i;
$digest = `md5sum $file`;
$digest =~ tr/[a-z]/[A-Z]/;
print "MD5: $digest\n";

For those of you who are not into writing Perl scripts, take a look a the first line that has a $digest in it. The shell command is between the ` `.  Feel free to replace the md5sum with anything else you want to run. You should remove the 2nd $digest line, unless you want the output in upper case.  Lastly, alter the print statement by changing the MD5 to whatever you want to call your plug-in. Save the file and place it in the plug-in directory.

Testing The Plugin

If you end up writing your own plug-ins, you can always test your plug-in by doing:

# ./{plug-in file name}.pl -i {test file name}

If the output is what you expect, then your ready to run it with the processor.

Please send me any plug-ins you write. I will be happy to include them with the download.

CSV Files

The CSV files that get generated with the --nudity and --stego options can be found under the directory tn.

nudity_review.csv

If you gave the Image processor the --nudity option it will create a file called nudity_review.csv. The contents contain the file name and score as per the Imager::SkinDetector. An example file looks like this:

/appl/scripts/data_carve_processors/test/data/Battlestar.jpg,9.43998543211
/appl/scripts/data_carve_processors/test/data/Nmap_Matrix_Screen_Huge.jpg,0.235281046114735
/appl/scripts/data_carve_processors/test/data/annie-360x184.png,23.9994074147233
/appl/scripts/data_carve_processors/test/data/annie-720x368.png,23.6893463389346
/appl/scripts/data_carve_processors/test/data/axm-v3-10-p25.jpg,14.7481563652296
/appl/scripts/data_carve_processors/test/data/nmap-matrix2log-cropped.gif,6.15041880852475
/appl/scripts/data_carve_processors/test/data/nmap-matrixhax0r3c.gif,0
/appl/scripts/data_carve_processors/test/data/nmap-matrixhax0r3c.png,0
/appl/scripts/data_carve_processors/test/data/nmap_matrix5.png,0
/appl/scripts/data_carve_processors/test/data/nmap_matrix6.png,0

stego.csv

The stego.csv file will get created when the --stego is given. Again, this contains the file name and output from StegDetect. Here is an example:

/appl/scripts/data_carve_processors/test/data/Battlestar.jpg : skipped (false positive likely)
/appl/scripts/data_carve_processors/test/data/Nmap_Matrix_Screen_Huge.jpg : negative
/appl/scripts/data_carve_processors/test/data/axm-v3-10-p25.jpg : negative