PTJ

Minimalistic database for the analyzing and storing of passwords. This project came to be out of the need to quickly analyze a large number of passwords in a short timeframe. As I, like most of you I’m sure, got tired of a simple ruby/python/perl/bash/<insert language> script against a text file, the idea of storing everything in a database came to be. I know, it’s nothing revolutionary.

I’m using DataMapper in order to provide a database-agnostic front-end for users to easily (well, somewhat easily) query and extract analytics against the entire database, or a subset of passwords within the database.

‘Tags’ are used as a way to categorize subsets of passwords. Think of them like you would tags on a blog post. You might categorize a subset as ‘public’, or ‘internal’, or perhaps ‘company_name’. The number of tags which can be assigned to groups of passwords is unlimited, and provides an easy way to ensure the wrong passwords are not analyzed. This comes in handy when you want to, say, do an analysis on every password you cracked on an internal network every month, quarter, or year.

The following stats of a given password are currently being stored in the Password table:

id - Sequential number used as the database key.
password - Plaintext password
pw_hash - Hash of the password, if known. 
created_at - Time of when the password was added to the database.
upper - Boolean value telling you if an upper-case character is present.
lower - Boolean value telling you if a lower-case character is present.
number - Boolean value telling you if a number is present.
special - Boolean value telling you if a special character is present.
size - Length of the password.
sequence - Sequence of characters found in the password. 
  u - upper-case
  l - lower-case
  n - number
  s - special
  Example: Password123! => ulllllllnnns

Any number of combinations of these values, along with the tags can be used to look at subsets of data.

Using PTJ

Setup

Grab the repo, or grab the gem. The only difference should be the path where everything is stored. You can find the path of the gem like so,

gem which ptj
/some/path/gems/ptj-0.1.0/lib/ptj.rb

Navigate to the ‘etc’ directory

cd /some/path/gems/ptj-0.1.0/etc/

Modify the configuration file to point to the correct database.

When this is complete, you can setup a new project as such:

rake db:init

In order to test that this was successful, attempt to get the count of all items in the database.

rake db:count
  Passwords:   0
  Tags:        0
  Time Taken: 0.03307 seconds.

Importing Data

There is a provided script which will allow you to import data into the PTJ database.

Running it, you can see the following options:

ruby scripts/import.rb -h

Usage: import.rb [opts] -f file|-p password
  -t, --tag TAGS                   Tags to be used to identify imported passwords (separated by a comma)
      --[no-]strict                Enable/Disable Strict Mode
  -f, --file FILENAME              File to import.
  -r, --parser NUMBER              File parser to use:
                                   1 - Password Only
                                   2 - Hash:Password
                                   3 - Count, Password
                                   4 - Something ::: Password ::: Something
  -p, --password PASSWORD          Password to import.
  -a, --hash HASH                  Hash to import (Use in conjunction with -p).
  -h, --help                       Show this message.

If you feel like doing it the hard way, feel free to view the source of this script. Honestly, it’s not terribly difficult. However, the script will allow you to see the status of the import via a progress bar.

Analying Data

Again, there is a handy, included, script which can be used to quickly generate an ‘analysis’ of a subset of passwords. It can be found in the ‘scripts’ directory as well.

ruby scripts/analyze.rb -h

Usage: analyze.rb [opts]
  -t, --tags TAGS                  Tags to be used to when querying passwords (separated by a comma)
      --max-size SIZE              Maximum size of the resulting passords
      --min-size SIZE              Minimum size of the resulting passwords
      --[no-]upper                 Query based on upper-case letters
      --[no-]lower                 Query based on lower-case letters
      --[no-]special               Query based on special charaters
      --[no-]number                Query based on numbers
  -h, --help                       Show this message

Generating Wordlists

Guess what? There’s another script that we can use to generate a customized wordlist, sorted by number of occurrances. This really comes in handy once the PTJ database becomes full of rich data, and we’re looking for the top ‘X’ passwords which meets a certain criteria. For example, let’s say that we’re on an internal penetration test, and we need the top 20 passwords which include a special character. We can simply run:

ruby scripts/generate_wordlist.rb -t internal --special

Other Queries

Other queries can easily be made into a script, or simply from within irb. In order to do this, you really should take a look at DataMapper’s documentation: datamapper.org/docs/

A few examples:

Querying all samples with the tag of ‘internal’:

PTJ::Tag.get('internal').passwords

Querying all samples with upper-case and lower-case characters with a tag of ‘internal’:

PTJ::Tag.get('internal').passwords.all(:upper => true, :lower => true)

Get a list of the top sequences used by passwords:

PTJ::Password.aggregate(:sequence, :sequence.count).sort{|x,y| y[1] <=> x[1]}.each{|x| p x}

Contributing to ptj

Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet
Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it
Fork the project
Start a feature/bugfix branch
Commit and push until you are happy with your contribution
Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

PTJ