Overview

Writing decoders for binary formats is typically tedious and error prone. Binary formats are usually specified in text documents that developers have to read if they are to create decoders, a time consuming, frustrating, and costly process.

While there are high level markup languages such as ASN.1 for specifying formats, few specifications make use of these languages, and such markup languages cannot be retro-fitted to existing binary formats. 'bdec' is an attempt to specify arbitrary binary formats in a markup language, and create decoders automatically for that binary format given the high level specification.

Bdec can;

• Allow specifications to be easily written, updated, and maintained.
• Decode binary files directly from the specification.
• Generate portable, readable, and efficient C decoders.
• Run under Windows & Unix operating systems

The bdec xml specification uses constructs based loosely on those found in ASN.1.

Getting started

1. Download the latest version.
2. Go through the tutorial.
3. Try writing a specification for your own file formats (referring to the documentation where necessary).
4. Save lots of time and pain by not writing and maintaining hand written buggy decoders.

Documentation

If you're just getting started, the tutorial takes you step by step through the different concepts used in bdec. Once you've got started, refere to the rest of the documentation as required.

Support

Any comments or questions can go to the group mailing list at bdec-project@yahoogroups.com. This is a moderated list, so please be patient if your question doesn't show up immediately.

Features

• It is easy to quickly write (and read) a specification
• No need to write a custom decoder (all error conditions are handled internally; decoding either passes or fails).
• The command line application allows visual inspection of problematic files.
• Unicode aware, little-endian / big-endian aware
• Decode to xml for rapid development (eg: xml representation of a fat bootsector)
• Specify fields down to the bit (eg: single bit flags)
• Generated decoders are human readable, easy to use, and compile without warnings.

License

The bdec library and all decoders generated by the specification compiler are available under the GNU Lesser General Public License (LGPL). In summary, this means;

• The decoders can be built and used in any application (including commercial closed source applications).
• If used in a closed source application, the decoder must be dynamically linked (eg: .dll in Windows, .so in Linux).
• The source code (and any modifications) to the decoder must be made available to anyone you distribute the decoder to.

Download

The easiest way to install bdec is to use python's easy_install application. Run:

easy_install bdec

to install the latest version of the bdec decoder and compiler. The source code links below can be also be used if you install the dependencies manually (currently pyparsing and mako).

• Version 0.6.2 (2010-02-02)

  Bugfixes:

  • Correctly decode sequence values in the instance decoder.
  • Correctly resolve references found in conditional & constraint expressions.
  • Fix exception when we reference an unknown entry in an expression.
  • Report missing reference errors from the entry that defined the reference.
  • Improve the error messages from data errors.
  • Don't throw an exception when choosing and not enough data available.
  • Decode sequence entries with both a min and a max.
  • Correctly handle conditional references.
  • Report errors from the entry opening line & column number, not the closing location.
  • Data objects now supports non-seekable files such as stdin.

See the CHANGELOG for past releases.

Similar projects

Some projects that perform 'generic' binary decoding include;

• Hachoir - Very powerful decoding library, where binary formats are specified in python code (using generators). Quite high level decoding, with complex binary files being decoded. Lots of file formats are specified (at least partially).
• Construct - Decodes data structures, formats are specified in python code without too much high-level intelligence. Not many formats available.

Projects that generate decoders from high level specifications;

• Asn1c Asn.1 compiler; generates C language encoders and decoders.
• Google protocol buffers - Define file formats in an easy to read text specification, and compile them to Java, C++, or Python decoders and encoders.