-atlas wandering-
Bloggorama for breaking things

\

• /(75)

Subscribe
Subscribe to a (RSS) feed of this weblog.

Archives


This Blog

atlasutils-2.2.19.tgz

Intro to Vivisect
Vivisect is an interactive, extensible, disassembly/emulation/collaborative-analysis tool. While it shares much of its funcionality with other existing tools like IDA Pro, Vivisect excels at programmatic analysis, not requiring a GUI. Some of the crazier fans of Vivisect use it most frequently from an interactive iPython prompt.

you can get vivisect from my fork at https://github.com/atlas0fd00m/vivisect

== Architecture layout ==
* Vivisect Workspaces
In Vivisect, everything centers around the VivWorkspace object (often referred to as vw). Most information for a binary or memory-object can be obtained through accessor-like methods in this class, and in order to get anything of programmatic value out of Vivisect, you start out with a VivWorkspace.

* Envi Architectures
All disassembly and architecture-specific things happen in Envi. Each architecture implemented in Envi includes an ArchitectureModule and often an Emulator. The ArchitechtureModule implements things like makeOpcode() and operand-accessor methods. ArchitectureModules also provide a getEmulator() method which should return a clean new architecture-appropriate Emulator (eg. IntelEmulator, ArmEmulator, etc... based on the Emulator class). To get the most out of Vivisect, get to know the ArchitectureModule and Emulator classes well.

* impemu
The Import-Emulation (impemu) helper toolset lives in vivisect/impemu/ currently. While Envi provides architecture emulators, impemu.WorkspaceEmulator provides a great deal of power on top of stock Envi Emulator's, including a nifty helper-method runFunction() which provides a good framework for analytical emulation.

* Analysis Modules
Most of Vivisect's power derives from various analysis modules. Located in vivisect/analysis/ these analysis modules perform functionality from identifying function calling convention, to updating the VivWorkspace object with symbol information from the file-parsing, to identifying SEH Prologs in PE files. Analysis modules come in two flavors: Overall- and Function-Analysis Modules. Overall-analysis modules simply provide a function analyze(vw). Likewise, function-analysis mods provide a function analyzeFunction(vw, funcva), providing the virtual-address of the function in addition to the relevant VivWorkspace object.

* Event Subsystem
The backbone of Vivisect analysis is the event subsystem. More on that shortly...


== Viv Startup and Initial 'auto-analysis' Processing ==
While VivWorkspace objects can be created by other Python apps, vivisect provides vivbin to handle the setup of several powerful methods of using vivisect. Reading through vivbin will show you the different options available and how they effect the creation of the VivWorkspace object. vivbin either starts Vivisect in GUI mode or in Batch mode, and allows for collaborative hacking using Client/Server modes. In addition to these primary modes of operation, special analysis modules (called Scalpels) can be specified from the command-line.

Upon creation of the VivWorkspace, the initial file is loaded using vw.loadFile(), which identifies the type of binary (eg. PE or ELF) and loads the appropriate file-parser module, which then adds all pertinent information from the binary file and updates the VivWorkspace (using the parseFile() method of the parser module).

By default, initial auto-analysis is performed to a new binary (ie. not specified with -w filename.viv). This is done in vivbin by calling vw.analyze(), in both bulk and GUI modes. vw.analyze() begins by grabbing available code/functions from export-information from the file-parser. It then analyzes all pointers found in the workspace and sets each as code, data, etc.... Finally, vw.analyze() loops through the analysis modules and calls their analyze() function. Aside from those specified from the command-line, analysis modules are loaded by the file-parsers and added to the VivWorkspace using vw.addAnalysisModule() and vw.addFunctionAnalysisModule().



== VivWorkspace Events ==
Vivisect is highly event-driven, which allows workspaces to be shared easily over the network. When vw.addAnalysisModule() is called, for instance, that method actually calls a threaded method called _fireEvent(), handing in the appropriate event code and the module-name parameter. _fireEvent() acts as the doorway to the event system, allowing Vivisect to do virtually anything to handle the event... including sending it over the network. Eventually, an event handler is most often called, such as _handleADDMODULE() in this case. Vivisect's event model is a very powerful component of the overall tool.

The event subsystem has become the core component of the Vivisect storage module as well as the collaboration subsystem. Starting at the initial analysis of a binary, every event -- including for example the addition of analysis modules, adding comments, marking locations as code, naming functions, etc... -- is logged and then written to the .viv file. In fact, the .viv file format (as of late 2010) has become basically a storage facility for these events. When a .viv is loaded, these events are refired through the event subsystem to achieve the same effect. Eventually, this will also allow for the inclusion of a long-awaited "undo" functionality. There's nothing like marking a location as code only to find out it was a set of pointers, or rename the wrong function!


== Collaborative Vivisect ==
For a long time, Vivisect has included network collaboration, originally through a Vivisect server and numerous Vivisect clients, and later in a more peer-to-peer fashion through the "Shared Workspace" functionality. This network collaboration is enabled by network-marshaling of events. When clients make changes, they are run through the local event subsystem. If the local Vivisect instance is a client of some other Vivisect server/shared workspace, those events are sent to the server to also be processed. When the server handles events, events are sent out to the appropriate clients where they also process the events. This allows for real-time collaboration, where one person changes a function name and the name is simultaneously changed in the entire collaborative group. New Vivisect clients, upon connection, pull the list of events for the VivWorkspace, thus synchronizing with everyone else. This is a far superior method of collaboration compared to other systems which frequently replicate the database file itself.


== Emulation ==
Emulation officially happens in the Envi Emulator object. Each Envi architecture is responsible for providing a subclass of envi.Emulator. These emulators handle individual opcode emulation, and provide necessary methods for accessing registers, etc.... However, a very powerful subclass of the IntelEmulator is vivisect.impemu.WorkspaceEmulator. Used for import emulation, this class comes with many of the more "trace-like" amenities, including runFunction(). grep through the source for WorkspaceEmulator and research from there.
Most emulation which takes place in Vivisect is currently focused around a given function. Calls to other functions (internal or linked to) are not followed by default.

Import Emulation (impemu) includes "cheaters" which allow the emulation to handle calls to library functions in a way that's helpful for the emulator. For example, calls to malloc() by win32 binaries allocate memory within the Vivisect memory object, and return a magic value that allows emulation to continue as if the system were running in a real operating system.



-=-=-=-=-=-=-=-=- HOW TO USE VIVISECT -=-=-=-=-=-=-=-=-=-
There are several ways you may wish to use Vivisect. The primary methods of using Vivisect are through the GUI (default), Bulk Analysis (-B), loading and executing Analysis Modules (-M), and through Client/Server collaboration. Of course if you swing that direction, you may wish to simply import vivisect;vw=vivisect.VivWorkspace();vw.loadFromFile("filename");vw.analyze() and get underway from interactive python.

Visi expects you to execute vivbin from the local directory. Because it's often used on Windows, the #!python doesn't really work, so he doesn't fix it to be #!/usr/bin/env python or the like.
While I wrote an install script a long time ago (setup.py), it did not deal with the glade config files necessary for the GUI, and it died on the vine. Unless you are willing to maintain your own setup, use this method.
On Linux (ubuntu) I prefer setting the python line in vivbin correctly (#!/usr/bin/env python), then symlinking vivisect-current to vivisect_######## (side by side directories), then symlinking vivisect-current/* (all the directories) into /usr/local/lib/python2.#/dist-packages/. That way, when I shift from one version to another, I simply recreate the vivisect-current symlink. On occasion this causes difficulties, like when Visi add's a new TLD to the vivisect_######## directory. It's easily fixed, but can cause issues. I've also run into an occasional issue, like early in the vivclient lifespan. Still, it makes it run from the path, instead of having to be in the right directory.


== Bulk Loading ==
Many times it makes sense to simply have vivisect complete the initial analysis pass without a GUI. This allows you to bulk-analyze several binaries at once without the overhead of graphical updates. Some Binaries, such as XUL.DLL, are just best analyzed in this way. XUL.DLL is one of the binaries which is used to gauge Vivisect's performance, as it has a guargantuan .text segment, and forces viv through the hoops, so to speak. XUL.DLL is best done on a 64-bit Python interpreter due to Pythonic memory limitations.
One thing to keep in mind is that Bulk Analysis will attempt to write the .viv file in the same directory as the original binary. If you do not have correct permissions to write into that directory, analysis will proceed but at the end (yes, after however long it takes) the writing of that .viv will fail and all your (machine's) labor will have been in vain.

$ python vivbin -B path/to/binary.exe



== Running Analysis Modules ==
The greatest power behind Vivisect is its ability to run external analysis modules. While this capability is not exclusive to Vivisect, the value of these analysis modules can be quite great, particularly when the Vivisect Workspace is used with an Emulator or a WorkspaceEmulator-based thingy.
Analysis Modules run from the commandline without a GUI. Output from the analysis module can be either sent to stdout/stderr, stored in some non-viv fashion, or stored in the VivWorkspace (and subsequently ".saveWorkspace()"d to store the information to disk) in a number of ways. See the section on Writing Analysis Modules for more details on output.

$ python vivbin -M findkickassvuln.py /path/to/thing.exe.viv



== Collaborative Modes ==
Vivisect has two different collaborative modes: Server mode and Workspace Sharing.
Once the clients are connected (either to a server or a peer shared workspace), both modes function basically the same way, which is nearly the same as standalone mode. The GUI doesn't change, the commands don't change. The only difference is that the event subsystem gets pumped over the network to update other nodes' workspaces in real-time. When you add a comment, the ADDCOMMENT event is sent to the server (or system sharing the workspace, assuming that's not you), and from there the event is pushed to all the other connected clients. Making a location a function, or naming the fuction, or naming a local or an argument, blah blah blah... all of it causes the same events.

With the 20110526 release of Vivisect, a problem occurring if the client/server were using incompatible versions was resolved with the addition of the Vivisect Client, or "vivclient". Vivclient basically connects to the server using Cobra's docode functionality, which causes the remote system to be first source for python modules. From that point on, the Vivisect modules are loaded and the client started. All of the Vivisect modules should be the correct version because they were loaded from the server itself.

Warning: This means that the server's entire Python library is available over the network. If you have sensitive scalpels in the Python path, they are able to be pulled remotely with no real authentication. This is a fine tradeoff for most research networks.


=== Client / Server Mode ===
The first method, as the name suggests, is old-school Client/Server. The Server system listens for clients, coordinates communication and does little else. In this mode, the .viv file(s) need to be pre-analyzed (-B), and stored on the Server machine (-S). Clients connect to the Server using the Vivisect Client (introduced in 20110526) "vivclient"


$ python vivbin -S (on the server)

$ python vivclient -h
Usage: python vivclient [options] [ |
Options:
-h, --help show this help message and exit
-L, --list List workspaces available on
-w WORKSPACE, --workspace=WORKSPACE
Startup and connect to on

$ python vivclient -L thing (lists the viv's available on the remote system
Connecting to: thing : 16500
Retrieving workspace list: (name, status, port)
vmstat.viv Shutdown None

$ python vivclient -w vmstat.viv thing
Requesting vmstat.viv from server thing
/usr/local/lib/python2.6/dist-packages/vivisect/parsers/__init__.py:14: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5




=== Shared Workspace Mode ===
Shared Workspace mode is started by an already running Vivisect instance using the menu item Share -> Share Workspace. The following should be printed to the Vivisect Console:
Sharing workspace...
Workspace Listening Port: 46068

The port is random and assigned by the OS. We'll use this one for this example.
In Shared Workspace mode, each .viv is actually running and is assigned it's own TCP port. Because of this fact, we do not require the "-w vivfile.viv" option. Simply connecting to the right port is enough.

From Vivisect Client:

$ python vivclient 192.168.72.196 46068
/home/atlas/hacking/V/vivisect_20110526/vivisect/parsers/__init__.py:14: DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5







== Vivisect Graphical User Interface (GUI) ==

The Vivisect GUI is split into three primary windows: Console, Navi, Data/Structures, and Memory.

Console - This is the main control window. The Menu lives here as well as the commandline (for directly typing Viv commands) and the Output window.

Navi - The Navigation wondow, providing Functions, Imports, and Exports for the given binary (each with its own tab)

Data/Structures - This window houses information about the details of the binary (section information, etc..), Strings, and the list of Vstructs that have been applied (Elf32Dynamic, Elf32Reloc, etc... are placed in this list during the Autoanalysis phase).


=== The Console and the Menu ===

The Console Output screen is very handy for tracking what's going on during some of the more obscure or detailed analysis tools. For example, the (currently beta) bin diffing analysis outputs the results from Workspace comparison into the Console Output screen. There is no marking of the Output screen, and you may not notice it until you save the workspace, at which point you will see output similar to the following:

Saving libform.so.viv...
...save complete! (0 sec)

The Output window is scrollable and should never lose information. Warning! This can be a memory problem if you are not careful... but good for analysis that just needs to have space. There is nothing worse than long analysis whose most valuable output is just lost off the scroll buffer.


The CLI interface allows you to interact with Vivisect at a more base level. Enter "?" into the CLI line and press enter and the following shows up in the Output window. Typing "?cmd" will provide specific help about any given command:

viv> ?

Documented commands (type help ):
========================================
alias codediff eval funcmeta maps names report symboliks
argtrack codepath exports imports mem pathcount save vampsig
binstr config filemeta loc memcmp python script xrefs
chat emulate fscope make memdump quit search

Undocumented commands:
======================
EOF funcgraph help


alias allows you to create new commands which unroll to form other commands. Typing alias by itself shows the list of current aliases
viv> alias
Current Aliases:

By default no aliases exist.

argtrack - Track input arguments to the given function by name or address.
binstr - Display a binary representation of the given value expression
chat - Echo a message to any other users of a shared workspace. Usage: chat oh hai! Checkout 0x7c778030
codediff - Analyze a second workspace and identify similar/identical functions between them.
codepath - Show any known code paths from the specified from address to the to address
config - Set/view various configuration settings for Vivisect
emulate - Create an emulator for the given function to step through the code
eval - Evaluate an expression on the CLI to show it's value. Usage: eval (ecx+edx)/2
exports - List the exports in the workspace (or in a specific file).
filemeta - List the metadata about the current viv file and binary
fscope - Enumerate Strings/Import-calls within the scope of one function down it's calling graph
funcmeta - List metadata about a given function (calling convention, args, name, etc...)
funcgraph - Draw a pretty 2D codeblock graph of the function
imports - Show the imports in the workspace (or potentially just one file)
loc - Display the repr of a single location by va
make - Create new instances of locations in the vivisect workspace (define a location)
maps - Display the memory maps for the current VivWorkspace
mem - View memory (with optional formatting and size)
memcmp - Compare memory at the given locations.
memdump - Dump memory out to a file.
names - Show any names which contain the given argument. "names printf"
python - Execute python code OR start an interactive python interpreter.
quit - duh
report - Fire a report module by python path name
save - Save the current workspace
script - Execute a python file (with expression extensions mapped in as locals)
search - Search memory for patterns (encoded, numbers, etc...)
symboliks - Use the new symboliks subsystem - fun. more later
vampsig - Generate a vamp signature string for the given function's first block
xrefs - Show xrefs for a particular location (to, from or both)

As you can see, Vivisect was written with a great deal of power baked into the CLI. I personally hardly use the CLI, except when needing functionality that hasn't found its way into the GUI yet. Normally, that's the greatest value of the CLI. It helps development, and lets us use functionality that is bleeding edge (like the "alpha" codediff function).



-=-=-=-=-=-=-=-=- Writing Analysis Modules -=-=-=-=-=-=-=-=-
There are several different types of analysis modules. There are those which should be a part of the core Vivisect analysis modules, and there are specialized modules (aka "scalpels") which perform some special analysis which don't make sense for inclusion in Vivisect (such as finding buffer overflows or formatstring-mishandling... yeah, let's share those with the rest of the Vivisect users... NOT!).

[] permanent link / /