New analysis code being developed for 2014 data format

New code is being developed by Daniel Arcaro with help from Eric Hazen and Dan Gastler to analyze the new/final data format documented here: AMC13CommonFirmwareProposal. Also Ted Laird has written a tool to convert to root format:

P.S. If you end up dumping the AMC13 monitor buffers, but would like
to convert the dumps to .root files with TTrees of
std::vector, so that you can use "ordinary" analysis
programs to analyze them, feel free to use this program:

Bug reports are quite welcome.  (to Ted Laird)

Finding the Data

See EricFindingCMSData for some help.

Converting Data

The two options for file types processed by this code are binary and root files. The code to switch from one to another or vice versa is here:

Binary to Root: (From Ted Laird)

  • The main tool for making badcoffee binary files is hcalraw/cpp/badcoffee.cpp. Usage is cat a.dat | ./badcoffee fed#.

Root to Binary: or

  • The dumpraw code is a cmssw analyzer and requires a cmssw environment. The rootdumper only requires root.
  • The rootdumper is built by running the Makefile in rootdumper/cpp.
    • Run the code using ./ --rootFile=HTB_007850.root --feds=928 --nEvents=101 --outputFile=data1.dat
  • The dumpraw analyzer can be built with the same directions as the CMSSW section below.
    • Run the code using cmsRun .../python/


Unpacker consists of three classes to parse AMC13 data format: FedEvent, FedBlock, and FedAMC. Various tools have also been created to utilize these classes and analyze the data. All code can be found here:

Class documentation:

Unpacker Build and Run Instructions

Check out and build: List of tools and their uses:
  • dump64: Prints the raw data to screen in hex with 64bits per line. Used to sort through data by hand.
    • use: $ ./dump64 file.dat (usually followed by | less)
  • dump: Main dump tool to parse events and check for errors.
    • use: $ ./dump file.dat -v 1 -e 1 -c
    • -v flag for amount of information about the event printed to screen
    • -c flag to turn CRC checking on or off
    • -e flag to print specific errors to screen

Root and Histogram Options

If root is installed can be made by $make root=1. dumptmp is identical to dump but also creates a root file with a histogram of the error array for easier visualization with large files.


The dump software has also been implemented in a cmssw analyzer to extract raw data from global/local root files and unpack it. This requires a cmssw environment (most likely lxplus). The code can be found here and .../trunk/cmsswtools. General instructions to make an analyzer can be found here:

Recipe to build the analyzer code:

$ cmsrel CMSSW_#_#_#    ($ scram list   if you want to see the version choices) (The version should match the one that created the root file. Use edmProvDump name.root to check the version.) 
$ cd CMSSW_#_#_#/src
$ cmsenv (equivalent: eval `scramv1 runtime -csh`)
$ svn co
$ svn co
$ scram b

To run the analyzer first set the parameters in the python file analyzer/parse/python/ The most important parameters (besides the file name) are the HcalTBSource vs PoolSource and the rootType string input. Change these to correspond to either a local or global root file. The python script is set to run over multiple files in a directory. To then run the analyzer:

$ cmsRun analyzer/parse/python/

All of this should be done using the same version of CMSSW as what the root file was produced with. To determine this use $ edmProvDump rootfile.root to see the CMSSW version number and other attributes of the file. There are also other edm functions that are quite useful (of course only on lxplus).

To Do List


  • Add a doxygen configuration file and "doc" target to makefile so one can do 'make doc' and get the doxygen output.

New Features


  • In FedEvent::PreParse the expression if(GetNAMC() > 0 || GetNAMC() < 13) is always true. I think you mean to use && instead of ||.
  • In FedEvent::PreParse there are two possibilities for size: a) 2 and b) 4...14. It should throw an exception if any other value is passed. The comments should reflect this.

  • Check for consistency of LMSEPVC bits through an entire event
    • MS should be "10" for first block, "11" for intermediate blocks, "01" for final blocks.
    • C should be 1 when MS="01".
    • EP should always be "11" I think
  • Check Blk_No, AmcNo, BoardID in block headers for consistency throughout event
  • Add option to dump entire AMC payloads

-- EricHazen - 04 Aug 2014

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf unpacker.pdf r1 manage 125.4 K 14 Aug 2015 - 16:00 DanielArcaro Documentation for classes in unpacker
Edit | Attach | Watch | Print version | History: r22 | r20 < r19 < r18 < r17 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r18 - 23 Oct 2015 - DanielArcaro
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback