Logout

GeminiTwo 10a
Powered by PmWiki

Trackermap and its use to store , access and visualize tracker data

Introduction

In the past the main visualization tool for monitoring was a histogram presenter that showed a set of histograms and tables updated regularly. The CMS tracker has 75 million readout channels organized in 16924 modules each one being a complete detector: its monitoring requires many thousands of histograms to be computed every few minutes. An histogram presenter is not enough, this kind of browsing monitoring results will be almost unusable for the CMS tracker due to the huge amount of histograms and will be used only in very special cases. For this reason a way to summarize the histogram information visually was searched for.

So, a specialized 2D representation of the tracker (called “Tracker Map”) was developed, such as a scatter plot where all modules are represented in a single screen. Since the single module is flat, we imagine to disassemble the whole tracker and to assemble it again on a flat surface putting the single modules in positions which are connected to their spatial position.

http://www.ba.infn.it/%7emennea/wiki/tkmapn.png

Data from each one of the data sources can be represented on this map. For example the single module can show coded with a color:

• number of dead channels from construction database
• total number of rechits hitting the module in the last 100 events readout
• result of a comparison between the last histogram and a control histogram

Holes or hot spots in the map can pinpoint detector problems. The map may be used to request more data by using an interactive interface.The trackermap is also a way to store data about the whole tracker in a single file. In fact the basic idea behind the trackermap is that of having a new level of abstraction where single module histograms are replaced by trackermaps. Instead of having to worry about so many histograms you just book and fill a few trackermaps. The software will take care authomatically of booking and filling (if necessary) all those single module histograms.

Simplified Tracker Geometrical View

In order to make more easy the implementation of the trackermap we have defined a simplified view of the Tracker that allows for the easy navigation of the tracker structure without entering in the intricacies of the local TIB,TOB,TEC,.. structure. The view is shown in the figure below and produces a numbering(also reported in the picture below) of the modules which is very easy to understand and doesn’t require any detailed knowledge on how the tracker is built. We define the layer has a set of rings. Rings are set of modules arranged as… rings in the case of endcap disks and … rings in the case of barrel cylinders. Some rings are made by stereo modules so there we have two modules : one numbered n , the other 100+n. A single module is identified by a unique id with the numbers giving its layer, ring and module position in the ring (dependent on phi). This Id has the advantage , in contrast to the Geom Det Unit Id, to be quickly understood by an operator that doesn’t know how the tracker is built.

http://www.ba.infn.it/%7emennea/wiki/geoview.jpg

http://www.ba.infn.it/%7emennea/wiki/geoviewnum1.png

How the trackermap is stored

The tracker map is stored in XML format. It is a list of 16924 lines like this:

 <polygon id=“3311116″ …..>

one for each module. The id value represents the number identifying module in the simplified geometrical view, it is computed in the following way: layer*100000*+ring*1000+module. The layer is a number between 1 to 43; ring number normally starts from 1 except for TEC; module is a number bigger than 100 for stereo module.

The other attributes present for each module are shown in the table below:

attribute name	attribute value	comments
detid	integer	module Geom Det Id
capvids	list of integers separated by comma and enclosed in parentheses	ids of fed channels connected to module
value	float number	value resulting from all calls to fill() method for the module
count	integer number	number of times the method fill has been called
onclick, onmouseover,onmouseout	Javascript function call	these fields are used by the Javascript interface
message	text string	value set by calling method setText:its value appears when the mouse is over the module
pos	text string	information about the module : its value appears when the mouse is over the module
fill	text string	color of the polyline representing the module:authomatically computed from field value unless set explicitily by method fillc()
points	list of point coordinates	describes the polyline representing the module

The choice of this format allows the immediate display of the file with a svg enabled browser like firefox (in fact for some of the attributes we use the SVG syntax). In addition this image can be made interactive by adding a Javascript interface. It is possible to add any other information by adding new fields with arbitrary names. All the information can be also handled by any xml processing software. The file size is around 5 MB but its size can be reduced to 1/10 of this by compressing it.

Reference frame

As image, the trackermap is a collage of 43 small images each one representing a layer. The layout of these images is shown below and is a grid depending only on two parameters xsize and ysize. Each layer is drawn using its local reference frame (also shown in the figure below) and then moved to the appropriate grid position.Note that in the trackermap there is no information about the side of the module where the readout electronics is present. In 3D some of the module are read from the front side , others from the back side. To deal with this fact you should imagine a module read by the back side, as being transparent, in such a way that you can see the signal also from the other side.

http://www.ba.infn.it/%7emennea/wiki/frame.png

Other views of the tracker: the readout view or Fedtrackermap

The data filled in the trackermap (we will call it from now on geotrackermap) can be represented in different ways taking in account how the tracker is wired. Cable connections to each module are used tipically to read the module, to control the module and to ensure the power supply. Each of these cabling can in princile be represented in a fedtrackermap (since the readout is done through fed channels), a fectrackermap(from the name “fec” of the electronics which enables the control of modules through control rings each one connected to six modules,… Until now only the fedtrackermap has been implemented but the interface is ready to include other views .

The readout view trackermap (fedtrackermap) has the following graphical representation:

http://www.ba.infn.it/~zito/cms/fedtrackermap/fedtkmap2.png

This is based on the following fed hardware layout. Columns represent racks containing each one 3 crates. Each crate has 18 slots containing each one a fed with 96 fed channels. Each crate in the fedtrackermap is represented in this way:

http://www.ba.infn.it/~zito/cms/tmapdemo/images/singlecrate.png

The fedtrackermap is represented by a XML file similar to that of geotrackermap but in this case a cell is one of the 96 channels in a fed. Channels are grouped in boxes representing the Fed and boxes in crates. Among the information contained in the file is the id of the module read. In a single cell you can fill a colour, a value or an arbitrary text. The cell is identified by two integers representing the fedId and fedChannel. It is also possible with a single call to method fill(int moduleid, float qty) to enter the same value in the 2 or 3 channels connected to the module moduleid.

Compression and conversion to other graphics formats

The Web interface of the trackermap contains around 100 files that allow to perform any kind of interactive manipulation. However in some uses (for example as dashboard display or simply to store tracker data) you don’t need this interface but only a single file containing the image. For this reason the Tracker Map class is provided with a method save and save_as_fedtrackermap that can export the trackermaps content in “jpg” “png” and “pdf” format. It is also possible to save the image as “bare” SVG without any Javascript added.

The png format is a small size file that can be quickly displayed by any browser (no need for plugin) and for this reason is ideal for a dashboard display since this doesn’t need interactive features. Many drawing programs like Adobe Illustrator can read SVG and help in postprocessing the image (for a presentation) or in converting to other formats. A free drawing program that handles natively SVG is Inkscape. Last but not least a freely available program called batik can view SVG and convert it to arbitrary resolution images by line commands like the following:

 java -Xmx256m -jar /home/zito/temp/batik/batik-rasterizer.jar -w 2000. -h 2000. trackeradc.svg 

Testing your browser

The svg and javascript code works with firefox 2.0 and later. Use the files in this directory to check your browser. For example tifviewer.xhtml. Be careful that the Web interface needs to open a detached window: for this reason you must enable the site “www.ba.infn.it” to open pop-up window in “Content” tab of Firefox preferences.

Note also that the Web interface to work needs the server to send the right headers i.e. “application/xhtml+xml” for the main page (with a name something.xhtml). If the server sends the wrong headers ,as for example does the server, cmsdoc.cern.ch than you aren’t able to use the Web interface. However accessing locally the same files solves the problem. This you can obtain either by copying the files on your desktop or doing a klog to access a remote file on afs store.

Implementation for monitoring

Two basic classes

To build the tracker map we have implemented two classes: TrackerMap and TmModule . TmModule contains all informations for each module. TrackerMap is instead responsible for creating the 16924 objects Tm Module necessary to store the tracker information. To access quickly these objects it builds also two C++ maps (see table below). It is also responsible for filling the trackermap and for printing it.

map name	value used to access module information	comments
SvgModuleMap	module id	accessed using layer*10000+ring*1000+module
IdModuleMap	module Geom Det Id

The simplest code to build a trackermap to count digis in each module is:

//Before processing first event
 tkMap = new TrackerMap("title"); 

  //loop on digi in event 
        int idmod = (*iu)->geographicalId().rawId();
        int ndigi = collector.size();
        tkMap->fill(idmod,float(ndigi));
  // After processing last event
     tkMap->printall(true,0.,0.,"tmapname");

(After program execution you should find around 100 files created : the one to open with the browser is “tmapname”viewer.xhtml) These two classes can be used both for offline analysis and for online monitoring inside DQM framework. They can be found in CMSSW/CommonTools/TrackerMap. Their use doesn’t require access to tracker geometry data. All information needed by the trackermap software is in fact obtained by using the special ascii file tracker.dat found in the data directory of CommonTools/TrackerMap.

This file is recreated each time the tracker geometry changes by using the VisReco/VisCustomTracker plugin of Iguana with a command :

untracked string saveTrackerdat = "true"

inside the following section of the cfg file :

service = VisConfigurationService
        { }

Instructions on how to run iguana with VisCustomTracker plugin. Note that this feature has been implemented from CMSSW 140.

The file tracker.dat is in fact a list of all tracker modules containing for each module three kind of data that will be eventually obtained in the future by using CMS services:

• The GeomDetUnitId for each module and its corresponding layer, ring and module number.
• The position in space of the module
• The name of the module (this is obtained unpacking the GeomDetUnitId)

Filling and storing the trackermap

The trackermap can be filled like a scatterplot by using the method fill(modulekey,quantity).To identify the module you use the modulekey that can be one of the key used in the two C++ maps reported in a previous table:

modulekey
layer,ring,module
GeomDetUnitId

Normally the color used to display the module depends on the accumulated value (call to print(true)) or on value/count (call to print(false)) unless the user calls the method fillc(modulekey,red,green,blue) that will override this mechanism. A white color indicated a count=0 (i.e. no calls done to fill() or fillc()). The default palette is the rainbow palette starting from blue until red . The minimum value is blue, the maximum red unless you set explicitily the minimum and maximum value calling the print() method with a range of values. You can also use a yellow-red palette by calling tkmap→setPalette(2) . To return to the default palette you just choose palette number 1.We are also planning the possibility of defining customized palettes.

The method setText(modulekey,string) can be used to store any string in the field message.

We haven’t yet defined a way to store other attributes for each module but this is trivial since it just means adding a string to each line defining a module.

Fedtrackermap implementation

To represent single fed channel a new class TmApvPair as been introduced. The filling of single cells representing fedchannels can be done with methods fill_fed_channel and fillc_fed_channel similar to the methods seen before for filling the geometric

trackermap. The fedtrackermap is created automatically if you put in the configuration file:

PSet TkmapParameters =
	{
	    untracked bool loadFedCabling=true
	}

If you don’t fill anything in the fedtrackermap cells, the program will use the values stored in the connected modules to draw them.

The code to fill each cell in the geotrackermap and in the fedtrackermap with a random colour is:

 edm::ESHandle<SiStripFedCabling> tkFed;
tkMap = new TrackerMap("title"); 
const vector<unsigned short> feds = tkFed->feds();
for(vector<unsigned short>::const_iterator ifed = feds.begin();ifed<feds.end();ifed++){
    const std::vector<FedChannelConnection> theconn = tkFed->connections( *ifed );
    int red = rand()%256;int green=rand()%256;int blue=rand()%256;
    for(std::vector<FedChannelConnection>::const_iterator iconn = theconn.begin();iconn<theconn.end();iconn++){
         kMap->fillc_fed_channel(iconn->fedId(),
			      iconn->fedCh(),red,green,blue);
         tkMap->fillc(iconn->detId(),red,green,blue);
        }
    }
 tkMap->printall(true,0.,0.,"tmapname");

When you call the method “printall” the fedtrackermap is automatically added in the web interface.

List of most important methods

method	parameters	comments
fill	modulekey,value	fill value for a module in geotrackermap
fillc	modulekey, color	fill color in a cell representing a module in geotrackermap
fill_fed_channel	fedid, fedchannel, value	fill value in cell representing fed channel in fedtrackermap
fillc_fed_channel	fedid, fedchannel, color	fill color in fedchannel cell
print	printflag, min, max, name	save geotrackermap content and web interface in a single file (obsolete)
printall	printflag, min, max, name	create all the files needed to implement the “offline” Web interface
printonline	none	create all the files needed to implement the Web interface for online DQM
save	printflag, min, max, name, width, height	save geotrackermap content in a single file in one of the four formats png, jpg, pdf and svg
save_as_fedtrackermap	printflag, min, max, name, width, height	save fedtrackermap content in a single file in one of the four formats png, jpg, pdf and svg
setText	modulekey, string	load a string of text for the module specified

Test image: check of module numbering scheme

The test directory in CommonTools/TrackerMap contains a program Tracker Geometrytest.cc which is run every time the trackermap code is changed. You find in this directory the images produced by the last test. The same directory contains a README.1st file with instructions on how to run this and other programs in the directory.

---

Note - The paragraphs that follow need major rewriting!

---

The qt interface

On top of the SVG trackermap the Qt interface is implemented by defining two new classes VisTrackerMapBox and VisTrackerMap. These new classes use the TrackerMap class with all its methods used to fill and print the trackermap in SVG format. In addition they provide a qt graphical interface that allows the interactive use of the trackermap directly in a C++ program. Apart from qt support these new classes don’t add new dependencies. The two classes are both qt widgets (QWidget) and their purpose is:

• VisTrackerMapBox is a graphical windows containing all the gadgets necessary to scroll,pick,resize and zoom the trackermap contained in the central window.When the user clicks on a module this class emits the signal moduleSelected(module_GeomDetUnitId). This signal can be catched by a slot in the program creating the trackermap and used ,for example, to retrieve and display the histograms about the module.
• VisTrackerMap - graphical window that uses the trackermap adding the qt methods needed by qt to draw the trackermap in the window (paintEvent)

Adding the qt interface

If you want to use the qt interface, then instead of creating directly an object TrackerMap you should:

/* Create a VistrackerMapBox*/
VisTrackerMapBox *tkMapBox= new VisTrackerMapBox (IgQtSite::selfFrom (page->workspace ()), "TrackerMap");
/* Display the qt widget */
tkMapBox->show ();
/*Get the trackermap pointer */
TrackerMap *tm = tkMapBox->getTrackerMap();

All methods described in the previous section are applied to tm with instructions like:

tm->fillc(moduleid,red,green.blue);

VisTrackerMapBox has in addition a method update() that can be used every time you want to repaint the trackermap in the graphical window and setPaintOptions(type,min,max) that has the same parameters of the print method for the TrackerMap and can be used with similar results before calling update().

Demo on qt interface use

The program VisTrackerMapApp in the test directory analyzes a root file containing histograms for each module and builds a “summary” trackermap with the possibility to click a module getting its histogram displayed. To try this program I have provided a root file with the Adc distribution for each module. Running the program with this file, you should obtain something like this Note that this demo can be run outside CMSSW and only needs qt and root.

The recipe to run this demo is:

cd CMSSW_1_3_1/src/
eval `scramv1 runtime -csh`
cvs co -r CMSSW_1_3_1 CommonTools/TrackerMap
cd CommonTools/TrackerMap/test
scramv1 b
wget http://cmsdoc.cern.ch/~gzito/Tracker.root 
../../../../tmp/slc3_ia32_gcc323/src/CommonTools/TrackerMap/test/VisTrackerMapApp/VisTrackerMapApp

The basic idea is to transform this demo in a C++ viewer of SVG trackermaps that can be used to import SVG trackermaps or root files producing the trackermap in Qt with a interface similar to what you have in the svg plugin.

Three use cases

Anyhow to make more easy the use of these two classes in practical cases we have built other classes. To present these classes we will consider 3 use cases:

1. Building a trackermap to display some quantity for each module (we use the trackermap as an histogram with entries for each module). Example here showing average ADC value for each module.
2. Create one or more histograms for each module, then process them and store the result of this processing in the trackermap. After processing the histograms are deleted. For example each histogram contain the Adc distribution in the module. The trackermap will store and show the adc mean. The rms could be seen in the message window, in addition to some error message if the histogram has problems .
3. Create one or more histograms for each module. Then proceed as in the previous use case but instead of deleting the histograms, trasform them in a module summary in png (containing the histograms + comments) and connect each summary with the module representing it.When the user clicks on the module the summary is shown. Example of trackermap implementing this use case.If you look in the directory containing the trackermap you’ll find a png image for each module. The name of the image is the ID of the module in the simplified view. For example 1001015.png contains the histogram of the module 15 in ring 1 , layer 1 (i.e. TEC -z).

The three use cases for offline monitoring

Here the implementation is straightforward. We only add some more classes that can be eredited by the user program and can already perform most of the functionality requested by the use case. An ORCA version of this code can be found in ORCA/TrackerMonitoring/SiStripMonFramework.

1. The class Tk Counter Off must be extended to implement this use case. This class contains the virtual method setValue(event,value,mod) to define for each event and module the quantity that must be filled in the trackermap. The class Tk Digi Counter is an example of how this works in practice.

The three use cases in DQM framework

In DQM framework the implementation is more difficult because we must consider the producer and the client part.

1. The producer should create two scatter plots layers vs modules . The first contains the accumulated value of the quantity to fill in the trackermap The second the number of times the fill is called.example of scatter plot used to represent the trackermap at the producer level . Provided we know how the modules are ordered in each layer, it is easy to write a client that will scan the two plots and fill the trackermap.
2. In this case we have to book all the 16924 histograms! Then the client should process them one by one and refresh them from time to time the trackermap on disk. Example of client building the trackermap. Here the only problem is to store the Geom Det Unit Id or the ID of the module in the histogram name or title in order to know to which module it is connected. But in this case it would be better if the producer itself would care of all these histograms. A possibility would be to think about a two level process. A second level producer which is client of the first level producer will take care of all these histograms. The client program would ignore everything about them and only get in some way the values of the fields value , count and message in the trackermap.

3. This is more or less like the previous case but in addition also the histograms must be saved from time to time.Here the only problem is that each stored histogram should have a unique URL in order to be accessed when the module is clicked. The easiest way to implement this (but perhaps not the most efficient) is to have a directory created for each trackermap and here write each histogram as a separate file with a name equal to the module ID. (See the offline implementation for an example) Also in this case a two level process would be of great help (see previous use case).

Implementation in Iguana (for monitoring and event dislay)

Geometrical Tracker View Implementation

The tracker view in Iguana is the same that is implemented in DQM: i.e. the tracker is made by layers made by rings of modules, but its implementation is a done with a hyerarchy of class containers that make a lot easier the navigation.

• VisCuCmsTracker is the complete detector defined as a container of subdetectors VisCuTkSubDetectors
• VisCuTkSubDetector is a container of VisCuTkPartDetector: barrel or endcap part. We consider separated endcap +z and encap -z;
• VisCuTkPartDetector is a container of layers VisCuTkLayer;
• VisCuTkLayer is a container of sub layers VisCuTkSubLayer: rings, rods and petals. Barrel cylinders are made from rings and rods; endcap disks from rings and petals.
• VisCuTkSubLayer is a container of modules;
• VisCuTkModule describes a single module.

The first five classes inherit VisCuTkComposite class that defines a vector container with basic interface to navigate the model. In addition all these classes inherit VisCuTkState class which completes the model interface adding new methods: to return the ID number of sub detector part virtual int getId(), to return true if sub detector part is visible virtual bool isVisible(), to change the selection state of tracker components virtual void setVisible(). The methods used to navigate the model are in part defined in the abstract classes: CuTkCompositeDetector and CutkState. Member functions of the CuTkCompositeDetector class relevant to tracker model navigation are: the components() method which returns the number of components of the detector part; the getComponent(i) method which returns a pointer to the component i. In addition we have the method getOwner() which returns the parent of the part that we are considering, getId() which returns the number identifying the part that we are considering, getName() which returns the name of the part (this one is defined only for layers). For example to list all modules of ring j of PXB 2 the code is following:

     CuTkSubDetector* subDetector = tracker->getComponent(1);
     CuTkPartDetector* partDetector = subDetector->getComponent(2);
     CuTkLayer* layer = partDetector->getComponent(2);
     CuTkSubLayer* sublayer = layer->getComponent(j);
     for(unsigned int m=0; m<sublayer->components(); m++ ){
      CuTkModule* module = sublayer->getComponent(m+1);
       cout <<"module"<<module->getId()
       <<"ring"<<sublayer->getId()
       <<"layer"<<layer->getName()<<endl;
     }

Use of the trackermap as graphics interface to select single modules.

The two figures below show how the trackermap is used to select modules. The first figure is in fact the well known r-z section representation of half tracker. Each bar corresponds to a ring as defined in the simplified tracker view and is used to select a complete ring. a complete layer can be selected by clicking on the 43 yellow small squares. The green squares open instead a window like in the second picture below. This is the layer representation in the trackermap and is used to select single modules in the tracker.

%center%http://www.ba.infn.it/%7emennea/wiki/trackerpartselection.jpg

Figure2 - Window for tracker parts selection.

%center%http://www.ba.infn.it/%7emennea/wiki/moduleselection.jpg

Figure3 - On the right the window for barrel layer modules selection. It represents the first layer of the outer barrel. On the left the window for endcap layer modules selection. It represents the first layer of the TEC starting from -z axis. The fifth ring consists of stereo modules.

This picture shows the result of the selection both for the 2D view (trackermap) and 3D view of the tracker.

Use of trackermap as event display (the overlayed mode)

 The trackermap in iguana appears as a 

2D graphics object implemented as a C++ API using Qt package for graphics. For this purpose we have implementented the following two classes stored in VisCustomTracker package in CMSSW VisReco sub system:

• VisCuTkMapWindow defines a window (Figure1) containing an object implemented in VisCuTkMap2D, for creating tracker 2D map.
• VisCuTkMap2D

The figure below shows the graphics interface with various capability : zooming, printing ,etc

%center%http://www.ba.infn.it/%7emennea/wiki/2dmapwindow.jpg

Figure1 - Window for 2D Visualization of tracker. Selected tracker components will appear in white frame. If more than one layer has been selected, the different layers are shown superimposed. The “all tracker” button allows to separate all layers, representing the whole tracker in a 2D map that can be examined by using the scrollbars. This map can be printed using the “print” button. “Sim Hits”, “Rec Hits” and “Digi” buttons allow to display in 2D map respectively simulated, reconstructed hits and digi. The “separated module/overlayed modules” button allows to represent modules overlayed or separated. The two different representations are shown in Figure2.

The trackermap in iguana can be used as a 2D event display in the tracker. For this use the “separated” module representation used for monitoring(where each module is represented by a polyline without crossing other modules, is complemented with a “overlayed” module representation that show exactly how the modules overlap in space. The difference between the two representations can be appreciated in the picture below.

%center%http://www.ba.infn.it/%7emennea/wiki/modules_os.jpg

Figure2 - Overlayed and separated representation of TIB 1 layer in the 2D map. In the overlayed representation the modules keep their relative position: so you can see clearly how rings overlap in space.But there is no way to see stereo modules and it is difficult to count them. In the ”separated modules” representation, each module is a different polygon and you can see clearly that the whole layer has double modules (the blue triangles represent stereo modules). Also, adjacent rings have different number of modules.

This Visual check of CMS tracker geometry contains many examples of use of the trackermap as event display.

Use of trackermap to show the integrated signal at the module and channel level.Zooming and panning of the trackermap.

If you use the separated representation then it is possible to show the integrated signal in each module. For example If we sum the signal coming from around 100 events with no pileup , we get the following map. But the overlayed representation of modules allows also to show the integrated signal at the level of the single channel. The figure below shows the integrated signal display produced by simhits in a endcap disk: i.e. the color of each pixel is proportional to the number of simhits that fall in the pixel position.

%center%http://www.ba.infn.it/%7ezito/cms/endcapacc.png

If you add to this feature the possibility to zoom to single modules, this means that we can check during data taking how single channels in a module are behaving.