Revisions¶
| Version | Date | Description | Author |
| 1.1 | 30/11/2020 | Created the document | Kiana Setoodehnia |
| 1.2 | 04/12/2020 | Revised the document | Kiana Setoodehnia |
| 2.0 | 05/12/2020 | Final version | Kiana Setoodehnia |
Introduction¶
During August 2020, a new pnCCD detector was delivered to European XFEL from PNSensor. The serial numbers for this detector are 205 (for top sensor) and 206 (for bottom sensor). It was commissioned from the \(11^{th}\) to the \(16^{th}\) of August. We could not obtain the calibration data during that week. Instead, the calibration data were acquired on August the \(20^{th}\) at 1.6 keV beam energy using aluminium fluorescence.
This report presents the summary of the results of a performance analysis of the new pnCCD commissioning and calibration data obtained in August 2020. An overview of the dark characterization, common mode characterization, CTI evaluation, gain calibration, and full well capacity determination for this detector is given in this document.
Dark Characterization¶
This chapter presents an overview of the characterization of the dark data obtained with the new pnCCD detector.
Nomenclature¶
Following the original nomenclature used for CCD analysis performed by Steffen Hauf at European XFEL for FastCCD and pnCCD notebooks, the pnCCD detector is defined to be a matrix of 1024 rows by 1024 columns. Here, every row is horizontal and every column is vertical. In other words, rows and columns are defined by axis \(= 0\) and axis \(= 1\) in standard PYTHON definition, respectively. Since the pnCCD readout is horizontal, charge is collected per row. Also, each pixel along a column is readout in parallel and at the same time.
Note
- The above nomenclature is not the same as that used by the PNSensor. They follow the standard CCD nomenclature and define the detector as a matrix of 1024 columns by 1024 lines, where their columns correspond to our rows and each line is the same as what we define to be a column. This manual uses our nomenclature instead of that used by the PNSensor.
Empty Frames¶
With the pnCCD notebooks developed at European XFEL, we normally read the data in chunks of 100 frames. It then follows that regardless of the pnCCD gain, if the duration of a dark run is less than 500 trains or frames, there will be empty frames at the end of the run. In this case, the number of empty frames will be:
where \(EF\) is the number of empty frames appearing towards the end of the run, and \(n\) is the actual number of acquired trains or frames. Fig. 3.1 shows an example of a case where the number of acquired dark frames are fewer than 500, and thus there are empty frames at the end of the run. Fig. 3.2 shows an example of having 500 trains for a dark run, and thus there are no empty frames.
Warning
- It is important to search for and exclude the empty frames prior to calculation of the offset; otherwise, the mean signal (averaged over number of frames) per pixel will be affected by the empty frames whose signal values are zero for all pixels.
An overview plot showing the last 100 raw (no corrections applied) dark image frames of run number 1 from proposal 2714. In this case the last 18 image frames are empty because the total number of frames in this run were 482 instead of 500. These 18 empty frames need to be excluded prior to calculating the offset.
An overview plot showing the last 100 raw (no corrections applied) dark image frames of run number 340 from proposal 900166. In this case, 500 image frames were stored in the raw data file and there are no empty frames at the end of the run.
pnCCD Offset¶
Offset is the same as the dark pedestal. Regardless of the pnCCD mode of operation, the dark offset per pixel is calculated as:
where \(N\) is the total number of frames acquired in the dark run, and \(Pixel\,Signal\) is the value (in ADU) for each pixel of the pnCCD detector. Therefore, the dark offset per pixel is the pixel signal averaged over the number of frames acquired in a dark run. Since pnCCD has 1024 \(\times\) 1024 pixels, a matrix of the offset per pixel for all these pixels is known as the offset map (see Fig. 3.3). To calculate the offset map, we normally only consider up to 500 frames (sequence 0) even if there are more frames stored in the raw data files, i.e., there are more than one sequences. The offset map is calculated after rejecting the empty frames. Generally, the offset map shows a horizontal stripe structure along the CCD readout axes (see Fig. 3.3). This is expected for a pnCCD-type detector with a horizontal parallel readout.
At the moment, the bad pixels (see BadPixels), including the area of the central hole, is not excluded prior to the calculation of the dark offset map. However, excluding bad pixels surprisingly changes the offset map only by a small amount (see Fig. 3.34). Of course, if there are too many bad pixels, the change in the offset map and its average value over all pixels may be a bit more noticeable. For example, the dark run number 361 from proposal 900166 has an average offset (over the entire detector) of 11522.78 \(\pm\) 809.22 ADU if we do not exclude the bad pixels. But this value is reduced to 11505.59 \(\pm\) 767.40 ADU after exclusion of all bad pixels. In this case, the detector was exposed to a high intensity of scattered X-rays for a relatively long time before taking the dark run. This caused a dramatic increase in the number of hot (noisy) pixels near the central hole. Consequently, excluding these bad pixels has a more noticeable effect on the acquired dark offset.
(Right) Offset map, and (left) histogram of the offset map for save mode of operation with bias voltage of \(- 270\) V and gain configuration of 1/1. Data are from run 1 in proposal 2714.
Save Operation Mode¶
The pnCCD detector has 3 modes of operation:
- Save mode
- Normal mode
- High Dynamic Range (HDR) mode
Each of these modes has a different bias voltage and is to be used for a different purpose (see pnCCD Operation Manual). In this section, we present the dark offset characteristics of the Save Mode of operation.
Table 3.1 summarizes the mean offset for the individual CCD quadrants and for the whole detector for different gain configurations in the save mode. Fig. 3.3 shows an example of the offset map and its histogram for this mode of operation. The histogram binning is performed such that each bin has 1 ADU.
| Gain | Detector Region | Offset | Operation Mode |
| 1/1 | Whole Detector | 11196.91 \(\pm\) 758.51 ADU | Save |
| Lower Left Quadrant | 11111.63 \(\pm\) 578.64 ADU | ||
| Upper Left Quadrant | 11597.80 \(\pm\) 685.09 ADU | ||
| Lower Right Quadrant | 10701.73 \(\pm\) 735.06 ADU | ||
| Upper Right Quadrant | 11376.50 \(\pm\) 715.13 ADU | ||
| 1/4 | Whole Detector | 10415.13 \(\pm\) 619.75 ADU | Save |
| Lower Left Quadrant | 10446.11 \(\pm\) 529.52 ADU | ||
| Upper Left Quadrant | 10693.92 \(\pm\) 543.75 ADU | ||
| Lower Right Quadrant | 10086.76 \(\pm\) 621.49 ADU | ||
| Upper Right Quadrant | 10433.72 \(\pm\) 622.23 ADU | ||
| 1/16 | Whole Detector | 10185.49 \(\pm\) 595.67 ADU | Save |
| Lower Left Quadrant | 10251.94 \(\pm\) 527.13 ADU | ||
| Upper Left Quadrant | 10425.33 \(\pm\) 520.04 ADU | ||
| Lower Right Quadrant | 9894.64 \(\pm\) 598.83 ADU | ||
| Upper Right Quadrant | 10170.06 \(\pm\) 604.67 ADU | ||
| 1/64 | Whole Detector | 10121.99 \(\pm\) 589.66 ADU | Save |
| Lower Left Quadrant | 10196.98 \(\pm\) 526.19 ADU | ||
| Upper Left Quadrant | 10352.53 \(\pm\) 514.16 ADU | ||
| Lower Right Quadrant | 9841.56 \(\pm\) 592.65 ADU | ||
| Upper Right Quadrant | 10096.90 \(\pm\) 600.25 ADU | ||
| 1/256 | Whole Detector | 10109.32 \(\pm\) 588.43 ADU | Save |
| Lower Left Quadrant | 10186.05 \(\pm\) 526.35 ADU | ||
| Upper Left Quadrant | 10338.04 \(\pm\) 513.12 ADU | ||
| Lower Right Quadrant | 9832.37 \(\pm\) 591.23 ADU | ||
| Upper Right Quadrant | 10080.83 \(\pm\) 599.49 ADU | ||
| 1/340 | Whole Detector | 10113.11 \(\pm\) 588.36 ADU | Save |
| Lower Left Quadrant | 10186.05 \(\pm\) 526.35 ADU | ||
| Upper Left Quadrant | 10338.04 \(\pm\) 513.12 ADU | ||
| Lower Right Quadrant | 9832.37 \(\pm\) 591.23 ADU | ||
| Upper Right Quadrant | 10080.83 \(\pm\) 599.49 ADU | ||
| 1/512 | Whole Detector | 10117.59 \(\pm\) 588.36 ADU | Save |
| Lower Left Quadrant | 10191.80 \(\pm\) 526.57 ADU | ||
| Upper Left Quadrant | 10350.57 \(\pm\) 510.58 ADU | ||
| Lower Right Quadrant | 9842.72 \(\pm\) 590.51 ADU | ||
| Upper Right Quadrant | 10085.29 \(\pm\) 601.35 ADU |
Normal Operation Mode¶
In this section, we present the dark offsets corresponding to the normal mode of operation. Table 3.2 summarizes the mean offset for the individual CCD quadrants and for the whole detector for different gain configurations in the normal mode of operation. Fig. 3.4 shows an example of the offset map and its histogram for the normal mode of operation. The histogram binning is performed such that each bin has 1 ADU.
| Gain | Detector Region | Offset | Operation Mode |
| 1/1 | Whole Detector | 12004.78 \(\pm\) 880.78 ADU | Normal |
| Lower Left Quadrant | 11671.47 \(\pm\) 639.61 ADU | ||
| Upper Left Quadrant | 12674.40 \(\pm\) 722.17 ADU | ||
| Lower Right Quadrant | 11350.65 \(\pm\) 745.44 ADU | ||
| Upper Right Quadrant | 12322.61 \(\pm\) 726.88 ADU | ||
| 1/4 | Whole Detector | 10630.91 \(\pm\) 635.43 ADU | Normal |
| Lower Left Quadrant | 10599.78 \(\pm\) 540.68 ADU | ||
| Upper Left Quadrant | 10970.12 \(\pm\) 574.14 ADU | ||
| Lower Right Quadrant | 10263.55 \(\pm\) 622.48 ADU | ||
| Upper Right Quadrant | 10690.19 \(\pm\) 617.60 ADU | ||
| 1/16 | Whole Detector | 10256.59 \(\pm\) 597.49 ADU | Normal |
| Lower Left Quadrant | 10306.39 \(\pm\) 528.65 ADU | ||
| Upper Left Quadrant | 10506.06 \(\pm\) 522.44 ADU | ||
| Lower Right Quadrant | 9957.12 \(\pm\) 598.52 ADU | ||
| Upper Right Quadrant | 10256.78 \(\pm\) 602.44 ADU | ||
| 1/64 | Whole Detector | 10153.17 \(\pm\) 589.63 ADU | Normal |
| Lower Left Quadrant | 10223.67 \(\pm\) 526.57 ADU | ||
| Upper Left Quadrant | 10380.26 \(\pm\) 517.21 ADU | ||
| Lower Right Quadrant | 9871.63 \(\pm\) 592.69 ADU | ||
| Upper Right Quadrant | 10137.12 \(\pm\) 598.79 ADU | ||
| 1/256 | Whole Detector | 10129.58 \(\pm\) 587.97 ADU | Normal |
| Lower Left Quadrant | 10204.55 \(\pm\) 526.44 ADU | ||
| Upper Left Quadrant | 10351.47 \(\pm\) 516.16 ADU | ||
| Lower Right Quadrant | 9874.09 \(\pm\) 591.32 ADU | ||
| Upper Right Quadrant | 10109.20 \(\pm\) 598.26 ADU | ||
| 1/340 | Whole Detector | 10131.78 \(\pm\) 587.70 ADU | Normal |
| Lower Left Quadrant | 10206.06 \(\pm\) 526.14 ADU | ||
| Upper Left Quadrant | 10355.08 \(\pm\) 514.56 ADU | ||
| Lower Right Quadrant | 9856.73 \(\pm\) 591.48 ADU | ||
| Upper Right Quadrant | 10109.27 \(\pm\) 598.82 ADU | ||
| 1/512 | Whole Detector | 10134.66 \(\pm\) 587.41 ADU | Normal |
| Lower Left Quadrant | 10208.14 \(\pm\) 526.18 ADU | ||
| Upper Left Quadrant | 10359.87 \(\pm\) 512.95 ADU | ||
| Lower Right Quadrant | 9861.04 \(\pm\) 590.79 ADU | ||
| Upper Right Quadrant | 10109.61 \(\pm\) 599.65 ADU |
(Right) Offset map, and (left) histogram of the offset map for normal mode of operation with bias voltage of \(- 400\) V and gain configuration of 1/1. Data are from run 338 in proposal 900166.
High Dynamic Range Operation Mode¶
In this section, we present the dark offsets corresponding to the HDR mode of operation. Table 3.3 summarizes the mean offset for the individual CCD quadrants and for the whole detector for different gain configurations in the HDR mode of operation. Note that the data corresponding to gain 1/4 presented in Table 3.3 are obtained from a dark run taken during a different day than those of other gains. This is why, the dark offset of this particular run is slightly different from those of other runs presented in Table 3.3. The dark run corresponding to gain 1/4 was taken after the pnCCD was exposed to high intensity for a while, and thus around the central hole region, there was a lot of hot pixles, which ended up increasing the dark offset value.
Fig. 3.5 shows an example of the offset map and its histogram for HDR mode of operation. The histogram binning is performed such that each bin has 1 ADU.
| Gain | Detector Region | Offset | Operation Mode |
| 1/1 | Whole Detector | 11522.12 \(\pm\) 811.40 ADU | High Dynamic Range |
| Lower Left Quadrant | 11395.99 \(\pm\) 614.13 ADU | ||
| Upper Left Quadrant | 11943.23 \(\pm\) 746.20 ADU | ||
| Lower Right Quadrant | 11028.56 \(\pm\) 823.88 ADU | ||
| Upper Right Quadrant | 11720.68 \(\pm\) 737.88 ADU | ||
| 1/4 | Whole Detector | 10585.31 \(\pm\) 661.45 ADU | High Dynamic Range |
| Lower Left Quadrant | 10695.45 \(\pm\) 644.18 ADU | ||
| Upper Left Quadrant | 10792.36 \(\pm\) 562.41 ADU | ||
| Lower Right Quadrant | 10314.27 \(\pm\) 704.65 ADU | ||
| Upper Right Quadrant | 10539.16 \(\pm\) 625.86 ADU | ||
| 1/16 | Whole Detector | 10221.26 \(\pm\) 596.75 ADU | High Dynamic range |
| Lower Left Quadrant | 10282.53 \(\pm\) 528.70 ADU | ||
| Upper Left Quadrant | 10456.20 \(\pm\) 524.82 ADU | ||
| Lower Right Quadrant | 9931.24 \(\pm\) 601.98 ADU | ||
| Upper Right Quadrant | 10215.05 \(\pm\) 603.32 ADU | ||
| 1/64 | Whole Detector | 10140.89 \(\pm\) 589.58 ADU | High Dynamic Range |
| Lower Left Quadrant | 10212.83 \(\pm\) 526.16 ADU | ||
| Upper Left Quadrant | 10365.37 \(\pm\) 517.42 ADU | ||
| Lower Right Quadrant | 9860.22 \(\pm\) 592.85 ADU | ||
| Upper Right Quadrant | 10125.12 \(\pm\) 598.94 ADU | ||
| 1/256 | Whole Detector | 10123.58 \(\pm\) 588.02 ADU | High Dynamic Range |
| Lower Left Quadrant | 10196.99 \(\pm\) 526.82 ADU | ||
| Upper Left Quadrant | 10345.92 \(\pm\) 516.28 ADU | ||
| Lower Right Quadrant | 9846.84 \(\pm\) 591.20 ADU | ||
| Upper Right Quadrant | 10104.55 \(\pm\) 598.06 ADU | ||
| 1/340 | Whole Detector | 10126.63 \(\pm\) 587.71 ADU | High Dynamic Range |
| Lower Left Quadrant | 10199.66 \(\pm\) 526.51 ADU | ||
| Upper Left Quadrant | 10350.36 \(\pm\) 514.33 ADU | ||
| Lower Right Quadrant | 9851.13 \(\pm\) 591.19 ADU | ||
| Upper Right Quadrant | 10105.38 \(\pm\) 598.83 ADU | ||
| 1/512 | Whole Detector | 10130.21 \(\pm\) 587.58 ADU | High Dynamic Range |
| Lower Left Quadrant | 10201.98 \(\pm\) 526.33 ADU | ||
| Upper Left Quadrant | 10356.42 \(\pm\) 512.86 ADU | ||
| Lower Right Quadrant | 9855.62 \(\pm\) 590.81 ADU | ||
| Upper Right Quadrant | 10106.83 \(\pm\) 599.68 ADU |
(Right) Offset map, and (left) histogram of the offset map for HDR mode of operation with bias voltage of \(- 470\) V and gain configuration of 1/1. Data are from run 99 in proposal 2720.
Regardless of the bias voltage, the mean offset value calculated across the whole chip slightly decreases with decreasing gain factor until gain 1/64 and then it seems to remian more or less constant in lower gains (see Fig. 3.6). Similarly, the standard deviation in the dark offset decreases with decreasing gain factor until gain 1/16 and then it seems to remian more or less constant in lower gains (see Fig. 3.7).
The distribution of pnCCD dark offset (averaged over all pixels) as a function of pnCCD gain.
The distribution of the standard deviation in the pnCCD dark offset (averaged over all pixels) as a function of pnCCD gain.
The pnCCD dark offset increases as the detector’s power output increases. The latter is the highest for normal mode of operation (see NormalMode) followed by the HDR mode (see HDRMode) and finally the save mode of operation (see SaveMode).
In some runs, a periodic signal is superimposed on top of the measured dark signal. See CommonMode for a more detailed discussion about this effect.
Offset Stability in Time¶
During the first user run with the new pnCCD, the detector was left ON over a few days in the save mode of operation. We power cycled the detector a few times to put it in HDR mode. However, with the dark runs obtained in save mode, we can still study the stability of dark offset as a function of time. Fig. 3.8 and Fig. 3.9 show the results of this stability test in save mode of operation for gains 1/1 and 1/256, respectively. Fig. 3.10 shows the stability of dark offset during half a day in HDR mode of operation for gain 1/256.
Stability of dark offset (averaged over all pixels) as a function of time for gain 1/1 in save mode of operation. The detector was power cycled before taking runs 64 and 181.
Stability of dark offset (averaged over all pixels) as a function of time for gain 1/256 in save mode of operation. The detector was power cycled before taking runs 71 and 183.
Stability of dark offset (averaged over all pixels) as a function of time for gain 1/256 in HDR mode of operation. The detector was not power cycled during these runs.
Offset Correction¶
To correct for the offset, one simply has to subtract the offset map from each individual illuminated image frame.
Common Mode¶
In the pnCCD, every 128 rows called CAMEX (see pnCCD Operation Manual) are connected to the same readout electronics. Therefore, every 128 rows share the same amplifier, etc. As a result, any voltage variation in the readout electronics would affect these rows in a similar fashion.
Common mode is a periodic signal that is superimposed on top of the measured dark signal. Its origin comes from the aforementioned voltage variation in the readout electronics that are shared between every 128 rows. The offset maps of pnCCD are modulated with common mode, whose amplitude changes from frame to frame and from column to column. The frequency of this modulation can change from run to run.
Schematics of each pnCCD chip. Figure courtesy of Markus Kuster.
pnCCD has 16 CAMEXs which are divided into 4 quadrants. In other words, charge is read out horizontally along the 1024 rows but each row is separated into two halves, and the readouts are located on the left and right sides of the detector. Consequently, every CAMEX consists of 128 rows by 512 columns. The pnCCD quadrants with their readout electronics are displayed in Fig. 3.11.
In order to calculate the common mode (in ADU), a few factors need to be considered:
- A common mode axis should be determined depending on the readout axis. Common mode axis should always be prependicular to the readout axis for a CCD analysis. This is because in each CCD, a certain number of row (column) readouts share the same readout electronics and they are all going to be affected similarly by the common mode. Therefore, to calculate and correct the common mode effect, one needs a snapshot of pixels that are read out in parallel (at the same time). In other words, one needs to look at the pixels that are being read out at the same time with the same readout electronics. The axis which provides such a snapshot is prependicular to the readout axis.
Common mode value (in ADU) is calculated for each image frame. Each image is devided into 4 quadrants consisting of 512 rows by 512 columns. A median of pixel signals is then calculated for all pixels per column, i.e., pixels whose column number is the same (see the hatched area). This is because the readout axis in pnCCD is horizontal, and therefore, the pixels per column are read out at the same time. To correct for common mode, each median value (in ADU) is then subtracted from all pixel signals in the corresponding column, i.e., the subtraction takes place for pixels whose column number is the same. Therefore, 512 median values (corresponding to 512 columns) per quadrant are calculated and each of them is subtracted from all 512 pixels in each corresponding column.
A common mode block size must be determined depending on how many readout rows (columns) in the CCD share the same electronics. For example, in pnCCD, a CAMEX is defined to be every 128 rows, which share the same readout electronics (see the grid area in Fig. 3.12). In each row, there are 512 pixels (corresponding to 512 columns in that quadrant). Therefore, a CAMEX is the ideal common mode block size for pnCCD.
Depending on the readout structure, it may be the case that every \(n\) row (column) readout in a CCD shows a strong correlation due to the common mode effect. Generally, such correlations should be studied in advance to the common mode calculation. To correct for this effect, a median of the pixel signals shall be calculated over every \(n\) pixels along the common mode axis. Finally, this median should be subtracted from all pixels along the common mode axis to remove these correlations. In the CCD notebooks developed at European XFEL, we have attributed a variable called stride for the definition of common mode. Stride is the same as \(n\) in the desription above. Thus, if stride \(= 10\), this means that the median is calculated taking into account the pixel signals of every 10 pixels along the common mode axis. This median is then subtracted from pixel signals of all pixels along the common mode axis.
To acurately calculate the common mode effect, one needs to use pixels with reasonable event statistics. A variable called
minFracis defined, which is the minimum acceptable fractional number of pixels that can be evaluated for common mode calculation.minFracparameter is used to reject common mode values calculated with too few pixels without events. A pixel is rejected from evaluation of common mode if:- it is marked in the bad pixel map (see
BadPixels). - its signal value (in ADU) is larger than 4 times the value of noise (in ADU) of that pixel, i.e., there is an event in that pixel for that frame.
- it is marked in the bad pixel map (see
If the number of usable (not rejected) pixels in a row (column) of \(N\) pixels is less than minFrac \(\times\) \(N\), the common mode is set to 0, i.e., it is not evaluated from that row (column). This is to prevent areas of the sensor with larger event numbers that skew the common mode distribution significantly. In other words, common mode correction is only considered sensible for sparse datasets. Because minFrac denotes the minimum fraction of usable pixels with respect to all pixels, it should have a value between 0 and 1. minFrac \(=\) 0 means that the common mode is evaluated for all rows (columns), depending on the common mode axis, except those that have NaN values due to the presence of bad pixels in that row (column). Only those rows (columns) are ignored in this case.
To acquire the common mode values (in ADU) for pnCCD, the following calculation is performed per image frame:
- The image frame is first devided into 4 quadrants each having 512 rows by 512 columns (see Fig. 3.12).
- Since the readout axis for pnCCD is horizontal (along the rows), the common mode axis must be vertical (along the columns). This is shown by the hatched area on Fig. 3.12.
- The common mode block size for pnCCD analysis is not defined to be the size of a CAMEX (128 rows by 512 columns). To increase the likelihood of having enough pixels without high intensity events to be used for common mode calculation, Robert Hartmann from the PNSensor company suggested to use the pnCCD quadrant as the common mode block size. This means that the block size is defined to be 512 rows by 512 columns.
- For pnCCD, we let stride (see above) be equal to 1, which means the median is calculated over all pixels along the common mode axis, i.e., no pixel is skipped.
- The
minFracparameter is set to \(0\) for dark runs. This was chosen after analyzing many dark runs with different operation modes and gains. Since the pnCCD detector is new and it does not have any permanently bad pixels, dark runs usually do not show a high percentage of bad pixels. Therefore, if a run is dark, we would use all pixels in each column for calculation of common mode. Of course, those columns that may have a bad pixel in them are ignored, and common mode is not evaluated for them as described previously. On the other hand, if a run is not dark and contains illuminated data, theminFracparameter is set to \(0.25\). This means that the number of usable pixels per column must be at least 128 in order to evaluate the common mode from that column.- Finally, per quadrant, a median of pixel signal is calculated for all 512 pixels per column, i.e., pixels whose column number is the same (see the hatched area in Fig. 3.12).
- Therefore, per quadrant 512 common mode values (in ADU) corresponding to 512 columns in that quadrant are obtained. Each of these values is the common mode for one of the columns in that quadrant.
For quadrant 3. |
For quadrant 0. |
For quadrant 2. |
For quadrant 1. |
Ffor quadrant 3. |
For quadrant 0. |
For quadrant 2. |
For quadrant 1. |
Histograms of the calculated common modes per column per quadrant for gain 1/1 are shown in Table 3.4 for normal mode of operation of pnCCD as an example. Prior to calculation of these common mode values, the dark data were already corrected for the offset. This is why these histograms are centered near zero.
With decreasing the gain factor (gains \(=\) 1/16, 1/64, 1/256, 1/340 and 1/512), these distributions get much narrower (with reduced standard deviations) and get more centered around zero (see Table 3.5). This may be due to the fact that the noise is dominated by the electronics for these gains. However, the distributions are not always Gaussian, particularly for quadrants 1 and 3 (see Table 3.5).
Fig. 3.21 shows, as an example, the common mode values per column versus column numbers for each pnCCD quadrant corresponding to gain 1/340 in HDR mode of operation. This plot is only the snapshot of a particular image frame, namely frame 245. The modulation of the baseline dark signal is rather obvious for this example. In contrast, Fig. 3.22 shows the same results but for a dark run with gain 1/1 in save mode of operation. It looks like the modulation of the baseline dark signal is not too much for the latter example.
Common mode distribution changes from frame to frame, from column to column and from run to run (see Fig. 3.23).
Common mode values (in ADU) calculated per column versus the column number (related to time, see FourierTransform) for each pnCCD quadrant is shown for the image frame 245 of dark run number 102 proposal 2720. Gain was set to 1/340. pnCCD was in HDR mode of operation. This run has a total of 311 acquired frames. The modulation of the baseline signal is visible for all quadrants.
Common mode values (in ADU) calculated per column versus the column number (related to time, see FourierTransform) for each pnCCD quadrant is shown for the image frame 245 of dark run number 1 proposal 2714. Gain was set to 1/1 and pnCCD was in save mode of operation. This run has a total of 482 acquired frames. The modulation of the baseline signal is negligible for all quadrants.
Common mode values (in ADU) calculated per column versus the column number for pnCCD quadrant 3 is shown for the two image frames of dark run number 103 proposal 2720. Gain was set to 1/512 and pnCCD was in HDR mode of operation. The results for frame 44 are shifted downwards by 15 ADU for visibility of both plots.
Common Mode Distribution¶
A Fourier transform of the time domain of the common mode values per quadrant will provide us with the frequencies at which common mode is observed. To obtain these frequencies, we first need to acquire the time domain of the common mode values per quadrant.
Time (in \(\mu\!\) s) to read a row per pnCCD quadrant is equal to:
where 512 is the number of pixels in a row per quadrant, and \(t\) is the time to read one pixel in the readout axis. The latter is determined by the pixel clock set by PNSensor. This is currently set to 29.2 \(\mu\!\) s, which is the time to read column-by-column pixels. Note that all rows in a quadrant are read out in parallel (at the same time). The pnCCD readout clock runs at 10 Hz. In other words, frame rate is 10 Hz given by FEL timing generator. Therefore, there is 100 ms (or 100000 \(\mu\!\) s) time between two successive images. It follows that the time (in \(\mu\!\) s) between when all rows (readout axes) per quadrant are fully read and when the next X-ray pulse hits the detector producing the next image frame will be:
where 512 is the number of pixels in a row per quadrant. Assuming that the first time-array corresponds to all rows per quadrant (each row consists of 512 columns, and all rows are read out in parallel at the same time meaning that pixels with the same position in all rows are being read out with identical timing) in an image, the next time-array (corresponding to the next image) should be concatenated to the first time-array with a 85049.6 \(\mu\!\) s gap in between the two time arrays. Also, the duration of each image should be 100 ms. If we then make a 1D time-array of all images per quadrant this way, we will have the time axis. The y-axis would contain the common mode values (in ADU) corresponding to each quadrant per image. If we plot y-axis vs. time, we will obtain the time domain of common mode values per quadrant. If we then make a Fourier transform of this distribution, we should see the frequencies of parasitic signals caused by the common mode.
It is worthwhile to mention that according to Robert Hartmann:
- Within 29.2 \(\mu\!\) s, the signals of all 128 channnels of one CAMEX are readout with a frequency of 200 ns. This is the frequency which we use as the sampling rate for our Fourier transform .
- The readout of one column takes 25.6 \(\mu\!\) s and there is \(29.2 - 25.6 = 3.6\) \(\mu\!\) s delay before the next column.
Two examples of the time and frequency distributions of the common mode for pnCCD quadrant 0 are shown in Fig. 3.24 for only the first frame, and in Fig. 3.25 for the first 10 image frames observed in that run. These examples are for gain 1/1 in save mode of operation. Fig. 3.26 shows the Power Spectral Density (PSD) for the first 10 image frames in the same run. PSD is the measure of common mode signals’ power content versus frequency.
Common mode time and frequency distributions obtained for the first image frame in quadrant 0 of pnCCD. Gain was set to 1/1, and the detector was operating in save mode of operation. Data are from dark run number 1 in proposal 2714. The time for acquiring one image frame is 512 \(\times\) 29.2 \(\mu\!\) s, where 512 is the number of pixels in a readout axis and 29.2 \(\mu\!\) s is the time to read one column in the readout axis, i.e., one pixel.
Common mode time and frequency distributions obtained for the first 10 image frames saved in the run. Each image frame is shown using a different color. This plot is only for quadrant 0 of pnCCD. Gain was set to 1/1, and the detector was operating in save mode of operation. Data are from dark run number 1 in proposal 2714. Here, the time includes the 85049.6 \(\mu\!\) s gaps in between the two successive image frames. Clearly, there are two frequencies that show up in each image frame with identical intervals. These are two peaks towards the beginning and end of each color band, and they may refer to the two clocks that keep repeating (29.2 \(\mu\!\) s and 85049.6 \(\mu\!\) s). The frequency amplitudes are normalized to the number of frequencies in the calculated frequency array.
Power spectral density for the first 10 image frames in dark run number 1 in proposal 2714. Gain was set to 1/1, and the detector was operating in save mode of operation. This plot is only for quadrant 0 of pnCCD.
Common Mode Correction¶
To correct for common mode effect, each of the 512 common mode values (in ADU) per quadrant are subtracted from the pixel signals (in ADU) for all pixels of the corresponding column, i.e., pixels whose column number is the same. Such a common mode correction applied per quadrant yields the best results in terms of homogeneity.
Correcting the common mode effect reduces the detector noise. Before going into a detailed discussion on noise (see Noise), it is worthwhile to mention that at high gains such as 1/1 and 1/4, the noise is entirely dominated by the CCD detector noise. Whereas, at lower gains (1/16, 1/64, 1/256, 1/340 and 1/512), the noise is dominated by the electronics noise coming from the FastADCs, etc. Table 3.6 shows an example of histograms of offset corrected dark signals prior to and after common mode correction for gain 1/1. An offset corrected dark signal shlould ideally be centered at zero. When we have common mode correction in addition to the offset correction, the full width at half maximum of the corrected signal is much reduced, which means the noise is reduced, and the result has a median that is more centered around zero as shown in Table 3.6.
|
|
Noise¶
As a crude estimation, the uncorrected noise map is a matrix of 1024 \(\times\) 1024 pixels whose values (in ADU) correspond to:
where \(N\) is the total number of frames acquired in the dark run, \(Signal\) is the value (in ADU) for each pixel of the pnCCD detector, and \(\mu\) is the dark offset value (in ADU) for the same pixel from the offset map, i.e., \(\mu\) is the mean (over all frames) of dark signal per pixel, which is the dark offset. Therefore, the uncorrected noise map is the standard deviation (over the total available trains or frames) from the dark offset per pixel.
The above description is only a crude estimation of the actual noise. Fig. 3.27 and Fig. 3.28 show examples of such an uncorrected noise map.
The uncorrected noise map essentially shows the standard deviation (over all image frames) in the dark offset per pixel. Data are from run 99 in proposal 2720. Gain was set to 1/1 and, the pnCCD was in HDR mode of operation.
Histogram of the uncorrected noise map. Data are from run 99 in proposal 2720. Gain was set to 1/1 and, the pnCCD was in HDR mode of operation.
Effect of Common Mode Correction on the Noise Level¶
A better estimation of the noise is achieved by correcting the dark data for offset (see OffsetCorrection) and common mode (see CommonModeCorrection).
Calculating the standard deviation of the offset and common mode corrected dark signal results in the first good estimate of the noise map. In other words, the noise map depicts standard deviation of the offset and common mode corrected dark signal per pixel, where the standard deviation is taken over the number of available image frames. Fig. 3.29 and Fig. 3.30 show examples of such a corrected noise map.
From this noise map, one can identify the noisy pixels (see BadPixels). Applying a mask on these bad pixels, the procedure of correcting the dark signal for offset, and common mode is repeated once again after masking the bad pixels. In addition, one can search for and exclude the cosmic ray events. Eventually, by calculating the standard deviation (over all image frames) of the final offset and common mode corrected dark signal, the final noise map is obtained. Fig. 3.31 shows the algorithm to obtain the final noise map, and Fig. 3.35 and Fig. 3.36 show examples of the final noise map with all bad pixels excluded.
Table 3.7 summarizes the final noise (averaged over all pixels) accross the entire detector for different gain and bias voltage configurations. Table 3.8 summarizes the average (over all pixels per quadrant) final noise for the individual pnCCD quadrants for different gain and bias voltage configurations.
Histogram of the noise map after applying offset and common mode corrections to the dark data. The standard deviation of the distribution is greatly reduced (in comparison with Fig. 3.28) due to the corrections. Data are from run 99 in proposal 2720. Gain was set to 1/1 and, the pnCCD was in HDR mode of operation.
After correcting the dark offset and performing a common mode correction on the offset corrected data, we obtain this noise map, which shows the standard deviation (over all image frames) in the corrected dark signal per pixel. Bad pixels are still included. This noise map is a better estimation of the actual noise. Data are from run 99 in proposal 2720. Gain was set to 1/1 and, the pnCCD was in HDR mode of operation.
Schematics of the algorithm used to calculate the final pnCCD noise map.
| pnCCD Gain | Uncorrected Noise | Corrected Noise (Bad Pixels Included) | Corrected Noise (Bad Pixels Excluded) | Operation Mode |
| 1/1 | 140 \(\pm\) 20 ADU | 95 \(\pm\) 23 ADU | 94 \(\pm\) 19 ADU | Normal |
| 1/4 | 36 \(\pm\) 15 ADU | 26 \(\pm\) 15 ADU | 25 \(\pm\) 5 ADU | Normal |
| 1/16 | 13 \(\pm\) 2 ADU | 11 \(\pm\) 2 ADU | 10.62 \(\pm\) 0.99 ADU | Normal |
| 1/64 | 9 \(\pm\) 1 ADU | 8.7 \(\pm\) 1.0 ADU | 8.7 \(\pm\) 0.5 ADU | Normal |
| 1/256 | 8.8 \(\pm\) 0.4 ADU | 8.6 \(\pm\) 0.4 ADU | 8.6 \(\pm\) 0.34 ADU | Normal |
| 1/340 | 8.7 \(\pm\) 0.4 ADU | 8.5 \(\pm\) 0.3 ADU | 8.4 \(\pm\) 0.5 ADU | Normal |
| 1/512 | 8.6 \(\pm\) 0.3 ADU | 8.4 \(\pm\) 0.3 ADU | 8.3 \(\pm\) 0.5 ADU | Normal |
| 1/1 | 116 \(\pm\) 12 ADU | 55 \(\pm\) 11 ADU | 55 \(\pm\) 6 ADU | High Dynamic Range |
| 1/4 | 34 \(\pm\) 9 ADU | 21 \(\pm\) 10 ADU | 21 \(\pm\) 6 ADU | High Dynamic Range |
| 1/16 | 12 \(\pm\) 4 ADU | 9 \(\pm\) 3 ADU | 9.3 \(\pm\) 0.7 ADU | High Dynamic Range |
| 1/64 | 9.0 \(\pm\) 0.5 ADU | 8.6 \(\pm\) 0.4 ADU | 8.62 \(\pm\) 0.38 ADU | High Dynamic Range |
| 1/256 | 8.7 \(\pm\) 0.4 ADU | 8.5 \(\pm\) 0.4 ADU | 8.54 \(\pm\) 0.37 ADU | High Dynamic Range |
| 1/340 | 8.6 \(\pm\) 0.4 ADU | 8.4 \(\pm\) 0.4 ADU | 8.45 \(\pm\) 0.37 ADU | High Dynamic Range |
| 1/512 | 8.6 \(\pm\) 0.4 ADU | 8.3 \(\pm\) 0.4 ADU | 8.34 \(\pm\) 0.37 ADU | High Dynamic Range |
| 1/1 | 111 \(\pm\) 13 ADU | 48 \(\pm\) 12 ADU | 47 \(\pm\) 8 ADU | Save |
| 1/4 | 29 \(\pm\) 9 ADU | 14 \(\pm\) 9 ADU | 14 \(\pm\) 1 ADU | Save |
| 1/16 | 11 \(\pm\) 2 ADU | 8.9 \(\pm\) 1.7 ADU | 8.8 \(\pm\) 0.6 ADU | Save |
| 1/64 | 8.8 \(\pm\) 1.1 ADU | 8.4 \(\pm\) 1.1 ADU | 8.4 \(\pm\) 0.5 ADU | Save |
| 1/256 | 8.6 \(\pm\) 0.4 ADU | 8.3 \(\pm\) 0.4 ADU | 8.3 \(\pm\) 0.5 ADU | Save |
| 1/340 | 8.5 \(\pm\) 0.3 ADU | 8.3 \(\pm\) 0.3 ADU | 8.2 \(\pm\) 0.5 ADU | Save |
| 1/512 | 8.4 \(\pm\) 0.3 ADU | 8.2 \(\pm\) 0.3 ADU | 8.1 \(\pm\) 0.5 ADU | Save |
| pnCCD Gain | pnCCD Quadrant | Final Noise | Operation Mode | Run/Proposal |
| 1/1 | Lower Left Quadrant | 48 \(\pm\) 2 ADU | Save | 1 / 2714 |
| Upper Left Quadrant | 48 \(\pm\) 2 ADU | |||
| Lower Right Quadrant | 49 \(\pm\) 2 ADU | |||
| Upper Right Quadrant | 46 \(\pm\) 2 ADU | |||
| 1/4 | Lower Left Quadrant | 14 \(\pm\) 1 ADU | Save | 2 / 2714 |
| Upper Left Quadrant | 14 \(\pm\) 1 ADU | |||
| Lower Right Quadrant | 15 \(\pm\) 1 ADU | |||
| Upper Right Quadrant | 14 \(\pm\) 1 ADU | |||
| 1/16 | Lower Left Quadrant | 8.8 \(\pm\) 0.4 ADU | Save | 3 / 2714 |
| Upper Left Quadrant | 8.9 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 9.1 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.8 \(\pm\) 0.3 ADU | |||
| 1/64 | Lower Left Quadrant | 8.4 \(\pm\) 0.3 ADU | Save | 4 / 2714 |
| Upper Left Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.6 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.3 \(\pm\) 0.3 ADU | |||
| 1/256 | Lower Left Quadrant | 8.3 \(\pm\) 0.4 ADU | Save | 5 / 2714 |
| Upper Left Quadrant | 8.3 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.5 \(\pm\) 0.4 ADU | |||
| Upper Right Quadrant | 8.3 \(\pm\) 0.3 ADU | |||
| 1/340 | Lower Left Quadrant | 8.2 \(\pm\) 0.3 ADU | Save | 6 / 2714 |
| Upper Left Quadrant | 8.2 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.2 \(\pm\) 0.3 ADU | |||
| 1/512 | Lower Left Quadrant | 8.1 \(\pm\) 0.3 ADU | Save | 7 / 2714 |
| Upper Left Quadrant | 8.1 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.1 \(\pm\) 0.3 ADU | |||
| 1/1 | Lower Left Quadrant | 81 \(\pm\) 13 ADU | Normal | 338 / 900166 |
| Upper Left Quadrant | 110 \(\pm\) 15 ADU | |||
| Lower Right Quadrant | 89 \(\pm\) 12 ADU | |||
| Upper Right Quadrant | 100 \(\pm\) 15 ADU | |||
| 1/4 | Lower Left Quadrant | 22 \(\pm\) 3 ADU | Normal | 339 / 900166 |
| Upper Left Quadrant | 29 \(\pm\) 4 ADU | |||
| Lower Right Quadrant | 24 \(\pm\) 3 ADU | |||
| Upper Right Quadrant | 27 \(\pm\) 4 ADU | |||
| 1/16 | Lower Left Quadrant | 10 \(\pm\) 1 ADU | Normal | 340 / 900166 |
| Upper Left Quadrant | 11 \(\pm\) 1 ADU | |||
| Lower Right Quadrant | 11 \(\pm\) 1 ADU | |||
| Upper Right Quadrant | 11 \(\pm\) 1 ADU | |||
| 1/64 | Lower Left Quadrant | 8.6 \(\pm\) 0.3 ADU | Normal | 341 / 900166 |
| Upper Left Quadrant | 8.8 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.9 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.6 \(\pm\) 0.3 ADU | |||
| 1/256 | Lower Left Quadrant | 8.6 \(\pm\) 0.3 ADU | Normal | 342 / 900166 |
| Upper Left Quadrant | 8.6 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.8 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.5 \(\pm\) 0.3 ADU | |||
| 1/340 | Lower Left Quadrant | 8.4 \(\pm\) 0.3 ADU | Normal | 343 / 900166 |
| Upper Left Quadrant | 8.5 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.7 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.3 \(\pm\) 0.3 ADU | |||
| 1/512 | Lower Left Quadrant | 8.3 \(\pm\) 0.3 ADU | Normal | 344 / 900166 |
| Upper Left Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.6 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.2 \(\pm\) 0.3 ADU | |||
| 1/1 | Lower Left Quadrant | 55 \(\pm\) 5 ADU | High Dynamic Range | 99 / 2720 |
| Upper Left Quadrant | 53 \(\pm\) 3 ADU | |||
| Lower Right Quadrant | 59 \(\pm\) 7 ADU | |||
| Upper Right Quadrant | 51 \(\pm\) 3 ADU | |||
| 1/4 | Lower Left Quadrant | 25 \(\pm\) 6 ADU | High Dynamic Range | 127 / 2714 |
| Upper Left Quadrant | 17 \(\pm\) 3 ADU | |||
| Lower Right Quadrant | 25 \(\pm\) 5 ADU | |||
| Upper Right Quadrant | 16 \(\pm\) 2 ADU | |||
| 1/16 | Lower Left Quadrant | 9.2 \(\pm\) 0.4 ADU | High Dynamic Range | 104 / 2720 |
| Upper Left Quadrant | 9.2 \(\pm\) 0.4 ADU | |||
| Lower Right Quadrant | 9.6 \(\pm\) 0.7 ADU | |||
| Upper Right Quadrant | 9.1 \(\pm\) 0.4 ADU | |||
| 1/64 | Lower Left Quadrant | 8.5 \(\pm\) 0.4 ADU | High Dynamic Range | 100 / 2720 |
| Upper Left Quadrant | 8.6 \(\pm\) 0.4 ADU | |||
| Lower Right Quadrant | 8.8 \(\pm\) 0.4 ADU | |||
| Upper Right Quadrant | 8.5 \(\pm\) 0.4 ADU | |||
| 1/256 | Lower Left Quadrant | 8.5 \(\pm\) 0.4 ADU | High Dynamic Range | 101 / 2720 |
| Upper Left Quadrant | 8.5 \(\pm\) 0.4 ADU | |||
| Lower Right Quadrant | 8.7 \(\pm\) 0.4 ADU | |||
| Upper Right Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| 1/340 | Lower Left Quadrant | 8.4 \(\pm\) 0.4 ADU | High Dynamic Range | 102 / 2720 |
| Upper Left Quadrant | 8.4 \(\pm\) 0.4 ADU | |||
| Lower Right Quadrant | 8.6 \(\pm\) 0.4 ADU | |||
| Upper Right Quadrant | 8.4 \(\pm\) 0.3 ADU | |||
| 1/512 | Lower Left Quadrant | 8.3 \(\pm\) 0.3 ADU | High Dynamic Range | 103 / 2720 |
| Upper Left Quadrant | 8.3 \(\pm\) 0.3 ADU | |||
| Lower Right Quadrant | 8.5 \(\pm\) 0.3 ADU | |||
| Upper Right Quadrant | 8.3 \(\pm\) 0.3 ADU |
Bad Pixels¶
The bad pixels map provides information about those pixels which should be permanently excluded from the data analysis due to their signal properties. This map is a matrix of 1024 rows by 1024 columns of pixels and is derived from dark data analysis (see Fig. 3.31). The following thresholds are applied frame-by-frame to identify such pixels:
- Offset Threshold: pixels whose signal values (in ADU) are:
where Offset refers to the offset map from dark data, median and standard deviation (\(\sigma\)) of the offset map are calculated over all image frames ignoring all possible NaN (not a number) values, and \(4\sigma_{O}\) refers to 4 times standard deviation of the offset, which is a limit considered at XFEL for CCD analysis based on recommendation from PNSensor company for bad pixels identification. Table 3.1 to Table 3.3 show the means and standard deviations of offset maps calculated per operation mode and pnCCD gain. To estimate signal values corresponding to bad pixels from the above mentioned equation, please use the Offset (BP Incl.) column and keep in mind that those are the mean offset values. Median values are up to 20 ADU higher.
- Noisy Threshold: Similarly, noisy pixels are those for which the rms noise of a pixel is:
where Noise refers to the rms noise map from dark data calculated accross the whole detector, and \(\sigma_{N}\) is the standard deviation of the rms noise. This is the final noise map after offset and common mode corrections are applied to the data. The areas/pixels identified by the above procedure, should be marked in the bad pixels map and are excluded from the data analysis.
- If there are possible cosmic ray events registered by the detector while taking data, the pixels whose signal values correspond to these events are identified and masked. These pixels are also marked in the bad pixels map and are excluded from the data analysis. The identification of cosmic ray events takes place by finding pixels whose signal values are:
where Signal is the offset and common mode corrected pixel-signal (in ADU), and Noise is the final rms noise map from dark data calculated accross the whole detector after offset and common mode corrections are applied to the data.
- Pixels in the central rectangular hole area of the pnCCD are also marked in the bad pixels map and are excluded from the data analysis. According to PNSensor, each pnCCD sensor (top and bottom detectors) has 22 rows and 60 columns cut out by a laser. Therefore, the central hole of the pnCCD starts from pixel coordinates \(=\) (489, 481), and ends at pixel coordinates \(=\) (533, 541), where the numbers in each parentheses set are the row and column numbers, respectively. The indices for counting rows and columns start from zero. This area is masked and is added to the bad pixels map.
- The edge pixels: according to PNSensor, the pixels on the first and last row and the first and last column should be marked as bad pixels and be excluded from the data analysis.
The number of bad pixels marked on the bad pixels map changes significantly from run to run and between different gain, bias voltage and X-ray intensity configurations. This effect can be mostly attributed to changing experiment and detector operating conditions. The bad pixels map is not static, i.e., each dark run results in a separate independent bad pixels map. The total number of bad pixles is at best around \(0.3\%\) of the entire number of pnCCD pixels and at worse may be well below \(5\%\) of the total number of pnCCD pixels. Table 3.9 shows an example of a bad pixels map acquired during commissioning of pnCCD in August 2020.
Effect of Bad Pixels on Offset Map and Noise Map¶
As mentioned before, normally the elimination of all bad pixels (see BadPixels) does not affect the offset map so much (see for example offsetSection). Masking bad pixels would of course reduce the standard deviation in the dark offset and possibly move its mean (over all pixels) to a lower value by a few ADU. Fig. 3.33 shows an example of statistics on the offset map with and without the bad pixels. Fig. 3.34 shows an example of the offset maps with and without bad pixels.
Elimination of the bad pixels from data before calculation of the final noise must be done. Fig. 3.35 and Fig. 3.36 show the heat map and histogram of the final noise map, from which our final noisy pixels could be determined. Fig. 3.37 shows an example of statistics on the final noise map with and without the bad pixels.
|
The logarithm of base 2 of the bad pixels map for gain 1/1 and HDR mode of operation. Data are from run 99 of proposal 2720. |
The effect of masking bad pixels on the statistics (over all pixels) of dark offset map. Data are from run 99 of proposal 2720, which is a dark run with gain 1/1. pnCCD was in the HDR mode of operation.
(Right) Offset map with bad pixels included, and (left) same offset map with bad pixels excluded for HDR mode of operation with bias voltage of \(- 470\) V and gain configuration of 1/4. Data are from run 127 in proposal 2714.
When we exclude all bad pixels from the dark data prior to correcting the dark offset and performing a common mode correction on the offset corrected data, we obtain the final noise map. This noise map is the best estimation of the actual noise. Data are from run 99 in proposal 2720. Gain was set to 1/1 and, the pnCCD was in HDR mode of operation.
Histogram of the final noise map after exclusion of the bad pixels and applying offset and common mode corrections to the dark data. The standard deviation of this distribution is further reduced (in comparison with Fig. 3.29) due to the exclusion of the bad pixels. Data are from run 99 in proposal 2720. Gain \(=\) 1/1 (HDR mode).
The effect of masking bad pixels on the statistics (over all pixels) of dark offset map and final noise map. Data are from run 99 of proposal 2720, which is a dark run with gain 1/1. pnCCD was in the HDR mode of operation.
Effect of Masking Bad Pixels on the Corrected Data¶
Eliminating bad pixels prior to offset and common mode corrections of the dark data has a noticeable effect on the final corrected signal. Namely, the standard deviation is further reduced (see Table 3.10).
Injecting Dark Constants to the Calibration Database¶
After processing each dark run (see pnCCD Operation Manual), three maps , which are so-called dark constants, are injected into the European XFEL calibration database. These maps are the offset, noise, and bad pixels maps. These dark constants could be retrieved from the calibration database when i) an illuminated run needs to be corrected and processed (see IlluminatedCorrection), and ii) for the online preview (see pnCCD Operation Manual).
Before injecting dark constants to the calibration database, each constant (offset, noise, and bad pixels maps) is tagged with the detector size (1024 by 1024 pixels), temperature in Kelvin, bias voltage, pnCCD gain, and a creation time. The latter is the time when the dark constants are successfully injected into the calibration database. This happens when the pnCCD dark analysis notebook is successfully ran to its end for a particular dark run and that run is processed such that the dark constants associated to that dark run are obtained.
If a dark run is processed twice and the dark constants were successfully injected to the calibration database on the first trial, they will not be injected again on the second trial. One has to ask the calibration team of the detector group to disable a set of dark constants associated to a particular dark run before one can inject them again with the same conditions.
During the retrieval process of dark constants, the calibration database is searched for conditions such as detector size, temperature, bias voltage, pnCCD gain, and creation time of dark constants. The calibration database then picks the constants with the desired conditions such that the creation time is closest in time to the time of retrieval of the constants, and the temperature could be any value within \(\pm\) 5 degrees Kelvin from the temperature used to tag the constants upon their injection to the calibration database.
Concluding Remarks and Recommendations¶
- We recommend that the dark runs, regardless of the pnCCD gain, are obtained for 500 trains, which would result in 500 frames. The number of trains can be counted using the sequencer counters. One can clear the sequencer, start the run immediately after and stop the run as soon as the counters show 510. Here, we assume that in between clearing the sequencer and starting the run, 10 trains are already counted. Taking the dark run for 500 trains or frames ensures that there are no empty frames at the end of the run. However, to be certain that the offset is not wrongly calculated due to possible empty frames, make sure that your algorithm searches for and excludes empty frames before calculating the offset.
- The stability of the dark offset as a function of time (without power cycling the detector) is not vigorously studied yet. This should be carried out in the future to examine whether or not the dark offset shifts over time if the detector’s conditions (such as bias voltage, gain and temperature) remain the same.
- Several dark runs were analyzed with different gains and bias voltages. As a result, sometimes we observe that the offset corrected dark signal is modulated with a periodic signal with an amplitude of up to 800 ADU (peak-to-valley) as shown in Fig. 3.22. The amplitude of the modulation changes from frame to frame and from column to column. This modulation originates in a periodic change of the baseline due to the common mode effect of the individual CCD columns. The frequency of this modulation can change from run to run. The distribution of this common mode is mostly (but not always) a Gaussian distribution, whose full width at half maximum could be over a couple of hundred ADUs broad depending on the gain factor of the pnCCD (see Table 3.4 to Table 3.5). With decreasing the gain factor, the full width at half maximum of the common mode distribution gets narrower and amounts to typical values of \(\approx\) 2 to 5 ADU and slightly changing from run to run. The applied column-by-colume common mode correction can partially compensate for the baseline variation in column direction.
|
|
- The pnCCD notebooks developed at European XFEL are capable of providing the time and frequency distribution of common mode values per quadrant in order to identify and remove the possible external parasitic frequencies.
- We do not apply an electronics saturation threshold (signal values \(\geq\) 65635 ADU due to 16-bit ADCs) for identifying bad pixels. This is because the CCD full well capacity (see
ChapterThree) limit will be reached well before the former saturation is reached. We analyzed a few dark and illuminated runs with different gains and operation modes and it does not look like the data show many pixels with signals above electronics saturation limit for any of the raw images used to calculate the offset and noise maps.
Characterization of Illuminated Data¶
When X-ray photons hit the pnCCD, some (or all) pixels register photon events resulting in illuminated images. To correct these images, we follow certain algorithms which are described in this chapter.
Data Corrections for Illuminated Runs¶
The first step for correcting the illuminated images is to search and remove any empty frames that may exist in the raw data. After that, we apply the bad pixels map to each raw image frame to mask any noisy pixel and events caused by cosmic rays, etc. The bad pixels map is obtained from a dark run taken with similar detector conditions (bias voltage, gain, temperature) as those of the illuminated run.
After applying the bad pixels map to the data, the next step is offset correction, which is done by subtracting the offset (obtained from the dark run) per pixel for all image frames (one at a time). The offset comes from the same dark run from which we used the bad pixels map. This dark run should preferably be taken close in time to the illuminated run to ensure that the offset correction is valid. Even though the detector conditions may be the same for the dark run and the illuminated run, if the offset varies by a bit over a long period of time, then the offset correction may not be accurate if the dark and illuminated runs are not taken close in time.
Warning
The pnCCD shows some afterglow behavior when it is illuminated by intense radiation. This makes it particularly important to take frequent dark runs in between illuminated runs to ensure correct offset subtraction.
All the aforementioned corrections explained above will always take place. What comes next depends on what additional corrections the users are interested in. The following corrections are optional and the users have the ability to turn them ON/OFF depending on their needs, as well as the availability of gain maps for photon energies of interest (see GainCorrection):
- Common mode correction
- Relative gain correction
- Split event correction and pattern classification
If common mode correction is desired, we perform a common mode correction per image frame right after offset correction. For the common mode correction step, we assume a minFrac of 0.25 (see commonMode), which means we need at least 128 usable pixels per column per quadrant to be able to perform common mode correction (see commonMode). Next, we examine the offset and common mode corrected signal per pixel per image frame to see if it is higher than a threshold. This threshold is set to 32000 ADU and comes from the full well capacity analysis (see FullWell). If the offset and common mode corrected signal per pixel per image frame is higher than this threshold, we mask the pixel with such a signal because most likely this event is not produced by any photons of interest. Masking such pixels also enusres that the common mode correction does not produce any undesired artifacts to the corrected data due to for example wrong choice of the minFrac parameter, in which case the large signals from photon events may skew the medians computed while correcting for common mode.
If relative gain correction is desired, the next step would be to correct the offset and common mode corrected data (provided that the common mode correction is turned ON) using the latest available relative gain map (see GainCorrection and Illuminated_Constants). The latter must be obtained for the photon energy of interest and detector conditions identical to the illuminated run. These conditions are bias voltage (i.e., the detector operation mode), pnCCD gain and temperature.
If relative gain correction is desired, it is performed by dividing the offset and common mode corrected data (provided that the common mode correction is turned ON) by the relative gain map per pixel for each image frame.
Note
Currently, we do not perform any charge transfer inefficiency correction to the pnCCD data because the charge transfer inefficiency is negligible for the pnCCD detector.
Finally, the last optional correction is split event correction and pattern classification, which is explained in detail in the next section. Before moving on to the next section, it is worth while to note that after each correction, we save the corrected data into a database with a separate path, and all these databases are saved in one corrected file. These files are found under the proc/ directory under each proposal directory.
To access the corrected data, one can open the separate databases mentioned above using the following paths:
- Raw data prior to any corrections:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/image
- Bad pixels map:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/mask
- Offset corrected data:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/pixels
- Common mode corrected data:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/pixels_cm
- Relative gain corrected data:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/gain
- Split event corrected data:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/pixels_classified
- The patterns resulting from the pattern classification algorithm:
/INSTRUMENT/SQS_NQS_PNCCD1MP/CAL/PNCCD_FMT-0:output/data/patterns
These paths are saved in files whose names are like CORR-R0354-PNCCD01-S00005.h5, where 354 is the run number and 5 is the sequence number (every 500 trains will construct one sequence of raw data saved to disk). These files are produced for any run which has been processed through Meta Data Catalog or by running the pnCCD correction notebook.
Split Event Correction and Pattern Classification¶
Photons impinging on a semiconductor detector will create an average number of electrons determined by:
where \(E\) is the energy deposited in the active sensor volume and \(\epsilon\) is a material specific constant determining the number of electron-hole pairs created per deposited energy unit. For silicon this constant is \(3.6\;\mathrm{eV/e^{-}}\).
In a CCD, the created charge drifts towards the readout structure according to the applied bias potential. During this drift time, latteral diffusion of the charge occurs, leading to charge being distributed over a non-negligable volume at the time of collection. This charge volume is frequently referred to as the charge cloud. Due to the cloud’s lateral extension of usually a few tens of micrometers, charge may be distributed over multiple pixels when the pixel size is small (as is the case for pnCCD). Each pixel will thus only register a fraction of the charge, corresponding to a fraction of the deposited energy.
Another effect responsible for broadening of the charge cloud is electrostatic repulsion which is described by Coulomb’s law. Diffusion and electrostatic repulsion have different relative importance on the final charge cloud diameter. We assume diffusion to be the dominating effect. Fig. 4.1 shows the example of charge sharing in one pixel. There are different areas defined for which charge sharing effect can be considered. The yellow circle represents the charge cloud. If the centre of the cloud is moved through the areas marked with \(D\), \(T\) and \(Q\), charge sharing with neighbouring pixels will occur. Different areas mean a different number of pixels affected by the charge cloud. \(D\) (doubles) represents events where charge is shared in two neighbouring pixels, for \(T\) (triples), three pixels are affected and for \(Q\) (quadruples), four pixels are affected. The center area labelled with \(S\) represents the area where there is no charge sharing effect and the generated charge cloud will be detected only in one pixel. Such events with no charge sharing are called single events. For singular photon events, i.e., events which do not result in overlapping charge clouds, the total distributed energy can be reconstructed by summing the charge in the event pattern’s pixel constituents. Charge sharing is evaluated using pattern classification algorithm.
Areas within one pixel specifed for charge sharing events. The charge cloud is represented by a yellow circle and its radius is labelled as \(r\). Note that this picture is made for the FastCCD detector, whose pixel size is much smaller than that of pnCCD. Therefore, the radius \(r\) does not indicate the size of charge cloud for pnCCD. If a photon hits the area labelled with \(S\), the resulting event pattern will be a single event. The same applies for the other areas, where \(D\) stands for doubles, \(T\) for triples and \(Q\) for quadruple event patterns. The image and caption are taken from Ivana Klačková master’s thesis p. 31).
The pattern classifier algorithm, developed by Steffen Hauf, uses an iterative algorithm to correct events which have resulted in charge sharing between multiple pixels. This is done in a multi-step process:
- The algorithm finds a pixel which has a value above a so-called primary threshold. This threshold is expected to be given in terms of \(n\times\sigma_{\mathrm{noise}}\), hence a noise map (from a dark run) also has to be supplied. The primary threshold is chosen to be \(4\sigma\).
- Within a 3 \(\times\) 3 or 4 \(\times\) 4 pixel region centered on the pixel explained above, the algorithm finds any pixels with values above a secondary threshold, given in terms of \(n\times\sigma_{\mathrm{noise}}\) as well. A 3 \(\times\) 3 region is used for single pixel events, a 4 \(\times\) 4 region is used for multi-pixel events. The secondary threshold is chosen to be \(3\sigma\). The number of pixels affected by charge splitting is referred to as an event pattern.
- The algorithm evaluates pixels above and below threshold pixels with respect to a list of i) valid patterns (see Fig. 4.2), which are pixel-charge distributions which can be created by a single impinging X-ray photon, and ii) invalid ones, which are distributions which could not have resulted from a single photon. Patterns are classified as either valid or invalid, depending on their shape. Only patterns with no direct neighbouring pattern are considered as valid. Patterns are assigned a multiplicity which corresponds to the number of pattern constituents. Valid single X-ray patterns have multiplicities 1 to 4 and the constituents form connected sub-parts of 2 \(\times\) 2 pixel square. Within each pattern, the charge distribution is additionally evaluated. For triples, the majority charge needs to be in the center pixel. For quadruples, the minority and majority charge must be diagonally opposite of each other. Valid patterns must be free-standing, i.e., no direct neighbour pixels may contain events. Diagonally offset pixels may contain events.
- If a positive match is found, the algorithm sums up the constituents and attributes the summed charge to the pixel with the majority charge. It negates all secondary constituents and values below threshold. If the summed value (in ADU) is below an upper threshold (which we assume to be 10 times the noise averaged over all pixels), pattern is considered valid. Otherwise it is attributed to a cosmic ray.
- The algorithm adds the pattern constituents to a pattern mask, indicating pattern multiplicity and orientation by an index. This pattern mask will also be used to avoid re-evalutating a pixel as part of other patterns.
Event patterns possibilities. Charge distribution amongst pixels is evaluated according to the displayed patterns. Each event is assigned a multiplicity with specific index as indicated in the plot. The image and caption are taken from Ivana Klačková master’s thesis (p. 46).
Schematic illustration of the first single events selection. Blue squares represent single events. Events marked with a red circle are considered as the first single events in the specified direction of readout defined by the direction of arrows. The image and caption are taken from Ivana Klačková master’s thesis (p. 46).
As a result of this algorithm, information about the total energy (in ADU) of the pattern and the pattern indices of the identified patterns are produced. The patterns which have no charge sharing with neighbouring pixels are marked as single event patterns. There is an option to select from the single events only those single events which are nearest to the readout for each row. Single events nearest to the readout are so-called first single events (see Fig. 4.3). Single events are used for charge transfer inefficiency and relative gain characterizations. As there are readout ports on both sides of the pnCCD sensors, right and left hemispheres of the pnCCD are analyzed separately.
Fig. 4.2 shows the distribution of the different valid event types: singles, doubles, triples and quadruples. Event occurance depends on pixel and charge cloud size and is thus detector and application specific.
On a well corrected and configured detector, events should be isotropically distributed, except for source profile effects, which should still affect all events of a given flavor similarly. Specifically, there should not be a preferred orientation for a given event flavor visible from the pattern plots.
For the split event correction and pattern classification, we use the bad pixels map to mask the bad pixels in the data. We also consider two hemispheres for the pnCCD: left and right. We then do pattern classification separately for the left and right sides of the detector.
pnCCD Gain Calibration¶
The gain calibration is performed using flat-field illumination of the pnCCD via aluminium fluorescence at 1.6 keV photon energy. It is of crucial importance that the flat-field condition is maintained; otherwise, the calibration is not worthy. Flat-field condition is reached when the entire surface of the detector can be illuminated homogeniously by the incident photons. Table 4.1 compares an ideal flat-field condition achieved in August - 2020 calibration of the new pnCCD with a non-flat-field condition achieved during May - 2019 calibration of the old pnCCD.
Flat-field illumination of the pnCCD is something that the SQS beamline scientists have to achieve by spending time on tweaking the beamline parameters to accomplish a homogenious illumination of the detector.
The only calibration data we currently have are achieved by aluminium fluorescence at 1.6 keV photon energy for gains and operation modes shown in Table 4.2.
Warning
Two very important rules that need to be strictly followed during data taking for calibration runs are:
- Beam must provide flat-field illumination of the pnCCD.
- Transmission must stay below \(10\%\) if the pnCCD is operated in normal mode because at higher transmission rates, the number of single events are drastically reduced, which ruins the statistics required for calibration of the pnCCD. This is particularly true for lower gains.
An image showing an ideal flat-field illumination of the new pnCCD during August - 2020 calibration. Data are from run 350 in proposal 900166. Gain was set to 1/4. Detector was running in normal mode of operation. This image is offset corrected and is the first image frame of the first seuence of the run. The pnCCD is homogeniously illuminated by the incident photons. |
An image showing a non-flat-field illumination of the old pnCCD during May - 2019 calibration. Data are from run 365 in proposal 900075. Gain is set to 1/1. Detector was running with bias voltage of \(-300\) V. As shown, the illumination is not homogenious. The incident photons are mostly only distributed around the center of the detector and the other parts of the detector was rarely seeing any photons. |
| pnCCD Gain | Photon Energy (keV) | Temperature Top Sensor / Bottom Sensor (degC) | Run | Dark Run | Proposal | Operation Mode | Transmission |
| 1/1 | 1.6 | -30 / -25.9 | 347 | 345 | 900166 | Normal | 9.37 |
| 1/4 | 1.6 | -30 / -25.9 | 350 | 349 | 900166 | Normal | 9.37 |
| 1/16 | 1.6 | -30 / -25.8 | 352 | 351 | 900166 | Normal | 9.37 |
| 1/64 | 1.6 | -30 / -25.8 | 354 | 353 | 900166 | Normal | 9.37 |
| 1/64 | 1.6 | -30 / -25.8 | 355 | 353 | 900166 | Normal | 50 |
| 1/256 | 1.6 | -30 / -25.7 | 360 | 356 | 900166 | Normal | 100 |
| 1/64 | 1.6 | -30 / -32 | 364 | 363 | 900166 | HDR | 20 |
| 1/256 | 1.6 | -30 / -31.6 | 366 | 365 | 900166 | HDR | 10 |
| 1/340 | 1.6 | -30 / -31 | 368 | 367 | 900166 | HDR | 50 |
Finding Singles Events¶
Once the calibration runs are corrected for offset and common mode, they are then corrected for charge split events and a pattern classification is performed for these corrected data (see Table 4.3, Fig. 4.8, and Fig. 4.9), which then identifies all valid events (see Fig. 4.10). Once the valid events for a particular run are identified, one can obtain their spectra by histogramming each of these pattern flavors. Fig. 4.11 shows an example of such spectra for only 500 image frames.
An example showing a raw uncorrected image. Data are from run 350 in proposal 900166. Gain was set to 1/4. Detector was running in normal mode of operation. This is the first frame out of 500 frames of sequence 0 of run 350. |
An example showing the image of single events after offset, common mode and split event corrections are performed. Data are from run 350 in proposal 900166. Gain was set to 1/4. Detector was running in normal mode of operation. This is the first frame out of 500 frames of sequence 0 of run 350. Distribution of single events are isotropic, which indicates an ideal flat-field measurement. |
Example of histograms of corrected and uncorrected raw data. Data are from run 347 of proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. This is one of the calibration runs taken in August - 2020. The noise peak centered around zero is cut due to the thresholds applied for pattern recombination analysis. The first peak, which appears around 5000 ADU, is the \(k_{\alpha1}\) line of alumimum at 1.486 keV. Other peaks are two-photons peak, etc.
Example of histograms of corrected data zoomed in. Gain correction improves the position of the peaks and make the peaks narrower. Data are from run 347 of proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. This is one of the calibration runs taken in August - 2020.
Example of statistics of valid events. LH and RH stand for left hemisphere and right hemisphere, respectively. Data are from run 347 of proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. This is one of the calibration runs taken in August - 2020.
Spectra of all valid events for run 347 of proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. This is one of the calibration runs taken in August - 2020. One sequence is equivalent to 500 image frames. The counts in the spectra are summed over the entire pnCCD detector. The first peak, which appears around 5000 ADU, is the \(k_{\alpha1}\) line of alumimum at 1.486 keV. Other peaks are two-photons peak, three-photons peak, etc.
We single out the single events and only work with those events for the rest of the analysis, i.e., charge transfer inefficiency and gain calibration. This is because the single events are simpler to deal with due to no charge sharing effects.
Per Quadrant Spectra¶
The spectra shown in Fig. 4.11 are obtained for the entire detector. However, pnCCD consists of two separate sensors (top and bottom sensors) each of which is divided into two parts such that each sensor has two readouts. Therefore, we are left with 4 quadrants and it would be better to acquire spectra per quadrant so we can examine the performance of each quadrant separately.
To obtain per quadrant spectra, we have two choices:
- Acquiring per row spectra for each quadrant, where rows are the readout axes. This means we will have 512 spectra per quadrant. These spectra could be the sum of all valid events or just a particular pattern flavor such as single events.
- Summing over all rows’ spectra per quadrant and obtaining one spectrum per quadrant, which is the sum of all individual spectra per row for each quadrant. Again, these spectra could be the sum of all valid events or just a particular pattern flavor such as single events.
Obtaining for each quadrant the per row spectra of single events is very useful in determination of the relative gain for each row (see GainCorrection). While, obtaining the sum (ove all rows) spectrum of single events per quadrant is used for determining absolute gain (eV/ADU) and the resolution (in eV) for each quadrant (see AbsoluteGain and Resolution).
Note
- Note that the readouts in pnCCD are parallel to the rows; therefore, all pixels in one row but different columns are connected to the same electronics, and thus have the same relative gain. The latter changes accross rows.
Fig. 4.12 shows an example of per row spectra of single events for a selected quadrant and Fig. 4.13 shows an example of the sum spectra of single events for all 4 quadrants from a gain calibration run. Depending on whether or not the relative gain correction is performed on the data, these spectra may or may not show peaks whose centroids are at the same position. If the relative gain correction is not performed, one would expect the per row spectra to have peak positions around a mean value due to small relative gain differences between each row (see Fig. 4.12). If on the other hand, the relative gain correction is performed, the spectra should show peaks whose positions are pretty much the same due to the relative gain correction (see Fig. 4.28).
Finally, one can also extract the spectrum of each pixel if this is desired (see Fig. 4.14).
Per row spectra of single events for lower left quadrant of pnCCD. These spectra are corrected for offset, common mode, and split event (charge sharing effect). But they are not yet corrected for relative gain. This is why, the centroid of the peak varies per row indicating small changes in relative gain from row to row. Data are taken from run 347 in proposal 900166. pnCCD gain was set to 1/1 and the detector was operated in Normal Mode. These data are from the August - 2020 calibration runs, and therefore, the peak observed in the spectra is the \(k_{\alpha1}\) line of alumimum at 1.486 keV.
Sum of all per row spectra of single events for each quadrant of pnCCD. The per row spectra for each quadrant are corrected for offset, common mode, and split event (charge sharing effect). But they are not yet corrected for relative gain. This is why, the centroid of the peak varies indicating small changes in an overall relative gain for each quadrant. Data are taken from run 347 in proposal 900166. pnCCD gain was set to 1/1 and the detector was operated in normal mode. These data are from the August - 2020 calibration runs, and therefore, the peak observed in the spectrum of each quadrant is the \(k_{\alpha1}\) line of alumimum at 1.486 keV.
The figure shows the spectrum of single events from randomly selected pixels. The position of each pixel (row \(\times\) column) is given on the top right corner of each graph. Data are taken from run 347 in proposal 900166. pnCCD gain was set to 1/1 and the detector was operated in normal mode. These data are from the August - 2020 calibration runs, and therefore, the peak observed in the spectrum of each pixel is the \(k_{\alpha1}\) line of alumimum at 1.486 keV.
Charge Transfer Inefficiency¶
Charge Transfer Inefficiency (CTI) is the fractional loss of charge during charge transfer in a CCD due to occurrence of traps in the pixel structure. For the evaluation of CTI, usually first single events are used. The reason for such an approach is that the charge of first single events has not been modified by shifting through charge remnants of the previous shifts left in lattice defects during the CCD readout process. This is because these events take place very close to the readout nodes, and thus charge has not gone through many pixels to reach the readout node. However, the first single events are not that many, so their statistics is usually small. Therefore, to increase the statistics, we at European XFEL use all single events (normal single events together with the first single events). Another reason why to use only single pixel hits is that these events do not need to be corrected for charge sharing, which, of course, introduces an uncertainty to the reconstructed event. To evaluate the CTI, all single events corresponding to the \(k_{\alpha1}\) peak of aluminum are plotted against their position in the columns for each readout axis (row) separately (see Fig. 4.15). These data are then fitted (see Fig. 4.16) by a linear function:
where the \(y\) values are the signal values (in ADU) corresponding to the single photon event and \(x\) values are the column numbers. The CTI value is represented by the following expression:
The CTI calculation algorithm allows specifying a range of values suitable for calculation. These values correspond to the minimum and maximum ADU value which define the photon peak region (in this case the \(k_{\alpha1}\) line of aluminum). The result of the CTI calculation is stored in an array containing one value of CTI for each readout axis (row), in total 512 values for one hemisphere of the pnCCD (see Fig. 4.17). This array is used for creation of a charge transfer efficiency (CTE) map (see Fig. 4.18) for the whole hemisphere as the CTE correction takes into account number of charge shifts during the readout procedure. The created CTE map can be used afterwards for correction of the CTI effects by multiplication of data and the CTE map on a per pixel basis. The CTE map contains values defined as:
where \(CTI_{(y)}\) is the value calculated by Equation (3.3) for each row of the pnCCD. The exponent \(x\) refers to the columns of the pnCCD. The value \(x\) represents the number of charge shifts needed to reach a readout node, which means \(x\) ranges from 0 to 512 for one quadrant of pnCCD. The resulting CTE map will be a 2D array with the same shape as one quadrant of the pnCCD detector. The CTE map is injected into the calibration database (see Illuminated_Constants); however, we do not currently use these maps for any data correction.
Note
- The CTI values for pnCCD are rather small and insignificant. This is attributed to the modern technologies for making silicon wafers. In modern detectors, the defects which cause charge traps in the silicon sensor are rather minimal and insignificant. When corrected, the CTE correction does not really have any noticeable effect on the images. We therefore ignore the CTE correction for pnCCD data.
- It could be the case that the fit of Equation (3.2) may not converge for a few columns such as the ones near the edges of the detector or those columns with a lot of bad pixels. Generally, if the single events statistics is poor in a column, the fit may not converge or may not be a good fit. Therefore, wherever the fit does not converge, we will set the corresponding relative gain to 1 and their CTI to the average value of the CTI in that quadrant. Generally, the number of non-convergant fits are somewhere between 4 to 17, which is a small percentange of all columns.
To evaluate the CTI, all single events corresponding to the \(k_{\alpha1}\) peak of aluminum are plotted against their position in the columns for each readout axis (row) separately. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. These data are from relative gain calibration data taken in August - 2020 at 1.6 keV photon energy. The \(k_{\alpha1}\) peak of aluminum shows up around 5700 ADU (as seen on the y-axis) when pnCCD bias voltage is \(-400\) V (normal mode) and gain is set to 1/1 (highest gain). These plots show the position of 100000 selected single events from many rows in pnCCD as a function of column number per quadrant. In other words, the integrated number of photons in the emission line (e.g., the \(k_{\alpha1}\) peak of aluminum) used to determine the relative gain and CTI at 1.6 keV is 100000 photons. The color shows the intensity normalized to the maximum value (ADU) and the red line is the average signal value (in ADU).
An example of the CTI fit for a particular row (row number 365) in a particular pnCCD quadrant (upper left). Here only 512 single events corresponding to 512 columns are shown. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. These data are from relative gain calibration data taken in August - 2020. The \(k_{\alpha1}\) peak of aluminum shows up around 5700 ADU (as seen on the y-axis) when pnCCD bias voltage is \(-400\) V (normal mode) and gain is set to 1/1 (highest gain).
An example of the CTI array (one value of CTI per row) per quadrant. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. These data are from relative gain calibration data taken in August - 2020. The \(k_{\alpha1}\) peak of aluminum shows up around 5700 ADU (as seen on the y-axis) when pnCCD bias voltage is \(-400\) V (normal mode) and gain is set to 1/1 (highest gain).
The CTE map derived for run 347 in proposal 900166. The gain was set to 1/1 and the detector was running in normal mode of operation. The charge transfer inefficiency is the most significant at the center of the CCD, which is the farthest distance away from the readout nodes.
Gain Calibration Method One¶
In a CCD, a relative difference in the amplification of charge takes place due to the different output amplifiers for each set of readouts. For example, in pnCCD, each CAMEX contain 128 readout rows. Therefore, these 128 readout rows share the same amplification. However, the next CAMEX have different amplifiers. Eventhough, ideally these amplifiers should be identical, in practice this is not achieved.
At European XFEL, the normal method of calculating the relative gain is from the linear fit used for estimating the CTI (Equation 3.2). Relative gain is defined as:
where \(b(i)\) is the intercept of the linear fit for row \(i\), and \(N\) is the total number of columns per quadrant.
One value of relative gain will be obtained per row (see Fig. 4.19). From these values, we can obtain a relative gain map per pixel basis (see Fig. 4.20). Such relative gain maps are injected into the calibration database (see Illuminated_Constants).
The relative gain per row calculated for gain 1/1 of normal mode. The data are taken from run 347 in proposal 900166. This gain map is obtained using method one described in GainCorrection.
The relative gain map for gain 1/1 of normal mode. The data are taken from run 347 in proposal 900166. This gain map is obtained based on method one described in GainCorrection.
Apart from the general method of obtaining the gain map described above, there is another way of acquiring the gain map from row-wise spectra. The latter method may be more accurate and is thus used by PNSensor as the method of obtaining their relative gain maps.
Gain Calibration Method Two¶
Once we obtain offset, common mode and split event corrected spectra for single events per row per quadrant, we can see peaks in these spectra (see Fig. 4.12). Depending on the FEL beam intensity during the calibration runs, we may be able to achive single photon zone, where only one photon hits each pixel, or we may be in multi-photons zone, where each pixel registers more than one photons. It is recommended to tune the beam such that the intensity allows single-photon zone because this would be a better calibration as identification of peaks are less troublesome. The peak observed in the spectra shown in Fig. 4.12 correspond to the \(k_{\alpha1}\) line of alumimum at 1.486 keV. If we zoom out, we will see two-photons, three-photons, etc. photo-peaks.
By fitting a Gaussian function to these photo-peaks for all rows, we can obtain another relative gain map if we subsitute the \(b(i)\) values in the numerator of Equation (3.5) by peak positions per row from the Gaussian fits, and by substituting the \(\sum_{n=0}^{N-1} b(i)\) values in the denominator of Equation (3.5) by the average peak position from all four quadrants (obtained from the Gaussian fits). The average (over all quadrants) peak position is used as a nornalization factor (hense, relative gain). A smaple of Gaussian fits is show in Fig. 4.21, and Fig. 4.22 shows the fit parameters obtained after fitting all the row-wise spectra of the lower left quadrant. These spectra are corrected for offset, common mode and split-events.
Example of Gaussian fits for 15 randomly selected rows from the lower left quadrant on pnCCD. The peak corresponds to the \(k_{\alpha1}\) peak of aluminum. The data are taken from run 347 in proposal 900166. Gain was set to 1/1 and the operation mode was normal mode.
Fit parameters obtained after fitting all the row-wise spectra of the lower left quadrant. These spectra are corrected for offset, common mode and split-events. \(A\), \(\mu\), \(\sigma\) refer to amplitude, centroid and width of each Gaussian fit, respectively. The goodness of fit is also shown on the \(\chi^{2}/dof\) plot. Data are from run 347 in proposal 900166 with gain 1/1.
Such a relative gain map is shown in Fig. 4.23. To compare this relative gain map with the one obtained from previously discussed method (see GainCorrection), see Fig. 4.24, Fig. 4.25, and Fig. 4.26. From the last two figures, one can see that the relative gain maps obtained from methods one and two are very much similar as the difference (one minus the other) between them is almost zero and the ratio of one over the other is almost one for almost all pixels.
The relative gain map for gain 1/1 of normal mode. The data are taken from run 347 in proposal 900166. This gain map is obtained based on row-wise spectra per quadrant (method two) described in GainCorrectionTwo.
The histograms of relative gain maps obtained from methods one (referred to as Old) and two (referred to as New). The data are taken from run 347 in proposal 900166. Gain was set to 1/1 and the detector was operating in normal mode of operation.
The difference (method one minus method 2 per pixel basis) between relative gain maps obtained from methods one and two. The data are taken from run 347 in proposal 900166. Gain was set to 1/1 and the detector was operating in normal mode of operation. The difference is zero most of the times, which indicates the two relative gain maps are very similar.
The ratio (method one divided by method 2 per pixel basis) between relative gain maps obtained from methods one and two. The data are taken from run 347 in proposal 900166. Gain was set to 1/1 and the detector was operating in normal mode of operation. The ratio is one most of the times, which indicates the two relative gain maps are very similar.
Validating Relative Gain Calibration¶
To validate the relative gain calibration, we have two options:
- One is to use the relative gain map (which can be obtained using method one, shown in
GainCorrection, or method two, shown inGainCorrectionTwo) and correct the data for relative gain. Then, obtain per row spectra of single events from the gain corrected data. These must show photo-peaks whose centroids are more or less the same because the difference in gain between readout rows is the cause of different peak positions for different readout rows. Such differences vanish when we correct for relative gain. If the photo-peak in these per row spectra are fitted using Gaussian functions, we should get a straight line if we plot position of the peak vs. row number.- Another method would be to use the relative gain maps from methods one (
GainCorrection) and two (GainCorrectionTwo) and correct the data for relative gain separately using these relative gain maps (one at a time), and then compare the final gain corrected single events spectrum (over the entire detector) which results from these two methods. Ideally, they should be identical.
The per row spectra of single events obtained from the gain corrected data are shown in Fig. 4.27 and Fig. 4.28, where gain correction is performed using the relative gain map calculated from method one and two, respectively. As you can see, the peak corresponding to the \(k_{\alpha1}\) line of alumimum at 1.486 keV lign up nicely for almost all rows in the two selected quadrants. The second peak in the spectra corresponds to the two-photons peak, and the position of this peak also ligns up nicely for every row in these quadrants. The peaks corresponding to the \(k_{\alpha1}\) line of alumimum in these spectra are separately fitted for each row using a Gaussian function, and the results are as expected: peak positions are almost identical for all readout rows (see Fig. 4.29 and Fig. 4.30).
The per row spectra of single events corresponding to the upper left quadrant of pnCCD. The spectra are obtained after relative gain correction based on method one (GainCorrection). Data are also corrected for offset, common mode and split events. Data are from run 347 in proposal 900166. Gain was set to 1/1 and the detector was in normal mode of operation. As can be seen, gain correction causes the peaks to lign up from all readout rows (shown by different colors), whereas without gain correction, the peak positions corresponding to different readout rows will not lign up (see Fig. 4.12).
The per row spectra of single events corresponding to the lower right quadrant of pnCCD. The spectra are obtained after relative gain correction based on method two (GainCorrectionTwo). Data are also corrected for offset, common mode and split events. Data are from run 347 in proposal 900166. Gain was set to 1/1 and the detector was in normal mode of operation. As can be seen, gain correction causes the peaks to lign up from all readout rows (shown by different colors), whereas without gain correction, the peak positions corresponding to different readout rows will not lign up (see Fig. 4.12). No matter which method we use to calculate relative gain map, the results are identical.
Fitting Gassuain functions to the first peaks observed in Fig. 4.27 results in peak positions that are almost identical for all rows (as can be seen from the middle panel on the top). This is because the spectra fitted are gain corrected (based on method one). When this is not the case, the peak positions are vastly different for different readout rows as can be seen in Fig. 4.22. \(A\), \(\mu\), \(\sigma\) refer to amplitude, centroid and width of each Gaussian fit, respectively. The goodness of fit is also shown on the \(\chi^{2}/dof\) plot. Data are from run 347 in proposal 900166 with gain 1/1.
Fitting Gassuain functions to the first peaks observed in Fig. 4.28 results in peak positions that are almost identical for all rows (as can be seen from the middle panel on the top). This is because the spectra fitted are gain corrected (based on method two). When this is not the case, the peak positions are vastly different for different readout rows as can be seen in Fig. 4.22. \(A\), \(\mu\), \(\sigma\) refer to amplitude, centroid and width of each Gaussian fit, respectively. The goodness of fit is also shown on the \(\chi^{2}/dof\) plot. Data are from run 347 in proposal 900166 with gain 1/1.
Comparing Method One with Method Two:
To compare the relative gain maps we have obtained using methods one and two, we decided to show a few factors. First, let us look at the single events spectra over the entire detector as a result of gain correction. This is shown in Fig. 4.31.
Single events spectra obtained for data of run 347 in proposal 900166. Gain was set to 1/1 and the mode of operation was normal mode. The peaks observed are the \(k_{\alpha1}\) line of alumimum (left peak), the two-photons peak (in the middle) and the three-photons peak (on the right). Regardless of the method used for calculating the relative gain map, relative gain correction makes the peaks sharper and reduces the widths of the peaks. It also changes the position of the centroids a little bit. It seems that methods one and two are almost identical. However, method two results in the narrowest peaks (best energy resolution).
Now, let us look at what happens to the map (per pixel basis) of peak position for the peak corresponding to the \(k_{\alpha1}\) line of alumimum. Fig. 4.32 and Fig. 4.33 compare the position maps derived after gain correction using methods one and two, respectively.
Position (in ADU) of the peak corresponding to the \(k_{\alpha1}\) line of alumimum. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the mode of operation was normal mode. Prior to obtaining this plot, the data are gain corrected using a relative gain map obtained from method one. As can be seen, the position remains more or less constant in each quadrant. However, different quadrants have different average positions.
Position (in ADU) of the peak corresponding to the \(k_{\alpha1}\) line of alumimum. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the mode of operation was normal mode. Prior to obtaining this plot, the data are gain corrected using a relative gain map obtained from method two. As can be seen, the position remains more or less constant accross the entire detector.
The two methods for calculating the relative gain maps have one distinctive difference, which is described below.
If we correct the data based on the relative gain map obtained from method one, and then obtain the single events spectra summed over all readout rows per quadrant, we get 4 spectra whose positions are not identical (see Fig. 4.34). In other words, it seems method one fails to correct for the difference between gains of each quadrant. However, if we do the same but we use the gain map calculated based on method two, then the sum spectra for single events per quadrant will have identical positions as shown in Fig. 4.35. This seems to indicate that method two is a better choice, which may be the reason this method is also used by PNSensor for their analysis.
Sum of single events spectra for all readout rows per quadrant. These spectra are gain corrected using the relative gain map obtained from method one. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the mode of operation was normal mode. Since these data are gain corrected, it is confusing why the spectra are not lined up. This indicates that somehow the relative gain map from method one fails to correct for the difference between gains of each quadrant.
Sum of single events spectra for all readout rows per quadrant. These spectra are gain corrected using the relative gain map obtained from method two. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the mode of operation was normal mode. Since these data are gain corrected, the spectra are lined up nicely as expected. This indicates that method two may be a better method to calculate the relative gain map.
Absolute Gain¶
To calibrate the peak positions (in ADU) to energy (in eV), we need to find out the absolute gain for pnCCD detector. This is done using the spectra of single events obtained for the entire detector, as well as those obtained for the individual quadrants. The latter spectra are the sum of all row-wise single events spectra per quadrant.
From the calibration data with aluminium target at 1.6 keV photon energy, the single events spectra (over the entire detector and for individual quadrants) always show at least 3 peaks. The first one corresponds to the \(k_{\alpha1}\) line of aluminum. The second and third peaks correspond to two-photons and three-photons peaks, respectively, where two- and three-photons peaks mean that pixels have been hit by two and three photons at once (multi-photon zone).
The fits of the three aforementioned peaks from the single events spectra of lower right quadrant are shown in Fig. 4.36 to Fig. 4.38 as examples for gain 1/1 in normal mode of operation. Fig. 4.39 to Fig. 4.41 show such examples for the spectra obtained over the entire detector.
Fitting a Guassian function to the \(k_{\alpha1}\) line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is summed over all row-wise spectra for lower right quadrant.
Fitting a skewed Guassian function to the two-photons line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is summed over all row-wise spectra for lower right quadrant.
Fitting a skewed Guassian function to the three-photons line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is summed over all row-wise spectra for lower right quadrant.
Fitting a skewed Guassian function to the \(k_{\alpha1}\) line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is the single events spectrum over the entire detector.
Fitting a skewed Guassian function to the two-photons line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is the single events spectrum over the entire detector.
Fitting a skewed Guassian function to the three-photons line of aluminum to derive its centroid and width in ADU. The data are from run 347 in proposal 900166. The gain was set to 1/1 and the pnCCD was operating in normal mode. The spectrum is the single events spectrum over the entire detector.
To obtain the absolute gain, we have to fit the aforementioned peaks using Gaussian or skewed Gaussian functions, depending on their shape, to acquire their centroids (in ADU) and their widths (in ADU). Knowing the energy of the associated peaks (because we have identified them as the \(k_{\alpha1}\), \(2\gamma\) and \(3\gamma\) lines of aluminum), we can convert the centroids’ ADU values to eV by plotting energies in (eV) vs. centroids (in ADU) and fitting a line with zero intercept through the data. The slope of this line is equal to the absolute gain (in eV/ADU). Table 4.4 shows the peak positions at each pnCCD gain and mode of operation.
Since the pnCCD is constructed from silicon and we know that the amount of energy required to make an electron-hole pair in silicon is 3.6 eV, we can divide the absolute gain (in eV/ADU) to 3.6 eV and obtain the conversion factor (in \(e^{-}\)/ADU). The conversion factor gives us the number of electrons released per ADU.
Examples of the linear fit, from which we obtain the absolute gain, and thus the conversion factor, are shown in Fig. 4.42 and Fig. 4.43 for gain 1/1 in normal mode of operation for lower right quadranrt and the entire detector, respectively.
Linear fit of energy (in eV) vs. position (in ADU) for the lower right quadrant. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the detector was operating in normal mode.
Linear fit of energy (in eV) vs. position (in ADU) for the the entire detector. The data are from run 347 in proposal 900166. Gain was set to 1/1 and the detector was operating in normal mode.
The absolute gains and conversion factors corresponding to each available calibration are provided in Table 4.5, which is in AbsoluteGains.
| pnCCD Gain | Operation Mode | Peak 1 Position (ADU) | Peak 2 Position (ADU) | Peak 3 Position (ADU) |
| 1/1 | Normal | 5730.4 \(\pm\) 0.3 | 11564.4 \(\pm\) 0.4 | 17419.3 \(\pm\) 2.1 |
| 1/4 | Normal | 1434.4 \(\pm\) 1.2 | 2787.7 \(\pm\) 1.2 | 4201.8 \(\pm\) 1.9 |
| 1/16 | Normal | 382.4 \(\pm\) 0.1 | 756.10 \(\pm\) 0.04 | 1149.4 \(\pm\) 0.2 |
| 1/64 | Normal | 99.4 \(\pm\) 0.1 | 195.2 \(\pm\) 0.1 | 289.3 \(\pm\) 0.2 |
| 1/64 | HDR | 98.7 \(\pm\) 0.1 | 196.6 \(\pm\) 0.1 | 292.6 \(\pm\) 0.1 |
Detector Resolution¶
Once we have the absolute gain, we can use it to convert the widths of the peaks to energy, and thus obtain detector resolution. The detector resolution for pnCCD specifically refers to the eV value of the width (\(\sigma\)) of the peak associated to the \(k_{\alpha1}\) line of aluminum. This can be calculated by multipling the absolute gain by the width of this peak (in ADU). The detector energy resolutions corresponding to each available calibration are provided in Table 4.5, which is in AbsoluteGains.
Available Gain Calibrations for pnCCD¶
The available pnCCD calibrations are provided in Table 4.5.
| Gain | Operation Mode | Region | Conversion Factor (\(e^{-}/ADU\)) | Absolute Gain (eV/ADU) | \(\sigma\) (eV) |
| 1/1 | Normal | Q0 | 0.08 | 0.27092 \(\pm\) 0.00043 | 70.5 \(\pm\) 0.2 |
| Q1 | 0.07 | 0.26808 \(\pm\) 0.00078 | 68.7 \(\pm\) 0.3 | ||
| Q2 | 0.07 | 0.26268 \(\pm\) 0.00143 | 71.9 \(\pm\) 0.4 | ||
| Q3 | 0.08 | 0.27101 \(\pm\) 0.00062 | 68.8 \(\pm\) 0.2 | ||
| ED | 0.07 | 0.25659 \(\pm\) 0.00053 | 71.1 \(\pm\) 0.2 | ||
| 1/4 | Normal | Q0 | 0.3 | 1.07004 \(\pm\) 0.00240 | 70.3 \(\pm\) 0.2 |
| Q1 | 0.29 | 1.06095 \(\pm\) 0.00279 | 69.5 \(\pm\) 0.2 | ||
| Q2 | 0.29 | 1.03510 \(\pm\) 0.01110 | 71.3 \(\pm\) 0.8 | ||
| Q3 | 0.3 | 1.06447 \(\pm\) 0.00663 | 68.7 \(\pm\) 0.5 | ||
| ED | 0.29 | 1.06109 \(\pm\) 0.00426 | 74.2 \(\pm\) 0.5 | ||
| 1/16 | Normal | Q0 | 1.15 | 4.13173 \(\pm\) 0.02915 | 82.6 \(\pm\) 0.8 |
| Q1 | 1.13 | 4.07806 \(\pm\) 0.00817 | 84.9 \(\pm\) 0.5 | ||
| Q2 | 1.11 | 4.00712 \(\pm\) 0.01914 | 85.1 \(\pm\) 0.6 | ||
| Q3 | 1.12 | 4.04413 \(\pm\) 0.04071 | 82.3 \(\pm\) 1.0 | ||
| ED | 1.08 | 3.88260 \(\pm\) 0.00169 | 101.0 \(\pm\) 0.4 | ||
| 1/64 | Normal | Q0 | 4.33 | 15.60126 \(\pm\) 0.08437 | 278.9 \(\pm\) 2.9 |
| Q1 | 4.19 | 15.07336 \(\pm\) 0.08352 | 276.3 \(\pm\) 3.0 | ||
| Q2 | 4.29 | 15.42615 \(\pm\) 0.07348 | 275.6 \(\pm\) 2.8 | ||
| Q3 | 4.22 | 15.20374 \(\pm\) 0.08210 | 274.3 \(\pm\) 2.9 | ||
| ED | 4.26 | 15.33222 \(\pm\) 0.07731 | 257.3 \(\pm\) 2.9 | ||
| 1/64 | HDR | Q0 | 4.35 | 15.64482 \(\pm\) 0.04932 | 269.1 \(\pm\) 2.9 |
| Q1 | 4.17 | 15.02896 \(\pm\) 0.04307 | 269.0 \(\pm\) 3.0 | ||
| Q2 | 4.28 | 15.41863 \(\pm\) 0.03752 | 270.6 \(\pm\) 2.7 | ||
| Q3 | 4.22 | 15.20813 \(\pm\) 0.04340 | 265.8 \(\pm\) 2.7 | ||
| ED | 4.22 | 15.19430 \(\pm\) 0.03813 | 247.1 \(\pm\) 2.7 |
Injecting Illuminated Constants to the Calibration Database¶
The available CTE maps and relative gain maps are injected to the calibration database of European XFEL everytime the pnCCD detector is calibrated using specific commissioning beamtimes. Regardless of which method is used to calculate the relative gain map, it can be injected into the calibration database. These relative gain maps, as well as the CTE maps, are tagged by a creation time, detector size (1024 by 1024 pixels), photon energy, bias voltage, pnCCD gain, and temperature (in units of Kelvin). Just like the dark constants (see Injection), a relative gain map with a set of tags (conditions) can only be injected into the calibration database once. Also, during the retrieval process of a relative gain map, the calibration database is searched for conditions such as detector size, temperature, bias voltage, pnCCD gain, photon energy, and creation time of the relative gain map. The calibration database then picks the constants with the desired conditions such that the creation time is closest in time to the time of retrieval of the relative gain map, and the temperature could be any value within \(\pm\) 5 degrees Kelvin from the temperature used to tag the relative gain map upon its injection to the calibration database. The relative gain map, as well as the CTI map (see CTI) are called illuminated constants.
Full Well Capacity of the pnCCD¶
Introduction¶
The amount of charge that can be accumulated in an individual pixel is defined by its full well capacity, and depends primarily on the pixel dimensions. Other design factors can also affect pixel charge capacity, including the requisite operational voltages. Saturation can be theoretically defined on the basis of full well conditions, but the complexity of potential well physics prevents simply considering well capacity as a performance parameter. It is common for CCD imagers to exhibit some effects of approaching a saturation condition before full well capacity is actually reached. As an example, the apparent sensitivity of a CCD may decrease with increasing average signal level because changes in charge transfer dynamics reduce the probability of electron trapping in potential wells nearing saturation. A possible result is increasing deviation by the CCD from linear response, even before blooming artifacts appear. In other words, if pixel saturation is considered to correspond to full well conditions, one early indication of its occurrence is the observation that increasing specimen illumination intensity does not result in additional signal detection; further charge accumulation may subsequently produce visible blooming artifacts. The quantitative capabilities of the CCD are compromised when its response deviates from linearity, and, as previously described, this may occur for some sensor designs before any visual effects of blooming are apparent. Blooming occurs when the charge in a pixel exceeds the saturation level and the charge starts to fill (or overflows) the adjacent pixels.
Certain anomalies in noise characteristics provide another indication of the approach to sensor saturation. Under illumination conditions in which the CCD is operating in the shot noise dominated regime, as light intensity and signal increase, random noise increases as the square-root of the signal. At some illumination level approaching the onset of full well, the positive random noise variations, which add to signal level, begin to be clipped, or smoothed. This phenomenon, termed noise clipping, causes the average noise level to begin to decrease, even while signal is increasing (see panel (a) in Fig. 5.1). At full well conditions, the noise decreases suddenly as charge is spread among neighboring pixels (blooming), while the signal remains stable. When Photon Transfer Curves (PTC) are constructed, plotting noise as a function of average signal, noise increases with signal at a slope governed by the dominant noise regime (see panel (b) in Fig. 5.1). Full well conditions, and the onset of blooming, results in an obvious break in the slope of the curve.
The Photon Transfer Curve (PTC) is a log-log plot of noise versus signal level, from which full well capacity can be determined. The diagram is taken from here.
To understand the plot shown in Fig. 5.1, three types of noise are defined:
- Read noise: This is represented by the first (flat) region of panel (b) of the graph in Fig. 5.1 and is the random noise that is associated with the CCD output amplifier and the camera’s signal-processing electronics. It tells you the minimum observed signal, and it is used to compute dynamic range. Read noise is directly measured from the PTC by recording the noise level at zero illumination.
- Shot noise: Shown in the second region of panel (b) of the graph in Fig. 5.1, it is inherent in the light itself. The noise does not originate in the camera. As the input light level increases in amplitude, the noise at the camera output rises out of the read-noise region and becomes dominated by shot noise. Shot noise is directly related to the input illumination and is proportional to the square root of that signal. The rms value of shot noise is equal to the square root of the mean number of photons incident on a given pixel. Thus, the shot noise profile becomes a straight line with a slope of \(1/2\) on the log-log plot.
- Fixed-pattern noise: The right-most region of the PTC represents the fixed-pattern noise, which becomes dominant at relatively high levels of illumination. This noise results from differences in sensitivity among individual pixels or from photo response nonuniformity. It is directly proportional to input signal strength.
As illumination levels are increased, the individual CCD pixels are unable to hold any additional charge without spilling over into adjacent wells (blooming). At this point on the noise curve, output noise abruptly drops because the signal value is clipped at the pixel’s maximum saturation level. At the point where the PTC peaks, as shown in panel (b) of Fig. 5.1, the CCD is said to have reached full-well capacity.
As described in Fig. 5.1, once the noise at each illumination level is calculated, a PTC is generated by plotting the camera’s output average standard deviation (rms of noise) vs. average signal level on a log-log plot. Then readout noise is directly available from the PTC by recording the noise level at zero illumination (readout noise is the point on the y-axis where background subtracted signal is zero). The full-well capacity parameter is the signal level at the point where the PTC drops. The conversion gain \(G\) of camera systems is typically expressed in \(e^{-}/ADU\). To obtain this gain, on the PTC plot, one should fit the shot noise section by a linear function (\(y\,=\,mx\,+b\)) using regression fitting method. \(G\) can then be found by extrapolation of the regression line towards the crossing with the horizontal axis. In other words, \(G\,=\,10^{-b/m}\) in \(e^{-}/ADU\). Full well capacity is derived from the maximum analog-to-digital output counts and the gain:
where \(S_{max}\) is the maximum analog-to-digital output (in ADU) before the PTC begins to slope downward, and \(FWC\) is the full well capacity in number of electrons. Finally, the dynamic range, which evaluates the ability of the camera to record low intensity signals alongside high intensity signals becomes:
where \(DR\) is the dynamic range expressed in dB, and \(\sigma\) is the read noise. Here, \(FWC\) and \(\sigma\) are both in units of number of electrons. Therefore, the limiting factors for the dynamic range are the full well capacity, which defines the maximum signal of the detector and the read noise of the detector which determines the lowest meaningful signal.
Obtaining Full Well Capacity of the pnCCD¶
In order to obtain the pnCCD full well capacity, we took some high intensity data during the commissioning of the new pnCCD detector in August - 2020. The runs are summarized in Table 5.1 to Table 5.4.
pnCCD Gain |
Photon Energy (keV) |
Pulse Energy (mJ) |
Temperature Top/Bottom (degC) |
Run | Dark Run |
Voltage Top/Bottom |
| 1/340 | 1.8 | 1.842 | -35 / -33 | 129 | 132 | -440/-450 |
| 1/340 | 1.8 | 1.842 | -35 / -33 | 130 | 132 | -440/-450 |
pnCCD Gain |
Photon Energy (keV) |
Pulse Energy (mJ) |
Temperature Top/Bottom (degC) |
Run | Dark Run |
Gas/ Solid |
Bunches |
| 1/256 | 1.8 | 1.94 | -35 / -29 | 164 | 170 | 1.4/11.4 | 50 |
| 1/256 | 1.8 | 1.94 | -35 / -29 | 166 | 170 | 2.9/11.4 | 50 |
| 1/256 | 1.8 | 2 | -35 / -30 | 167 | 170 | 1.4/11.4 | 25 |
| 1/256 | 1.8 | 2 | -35 / -30 | 168 | 170 | 1.4/11.4 | 15 |
| 1/256 | 1.8 | 2 | -35 / -30 | 169 | 170 | 1.4/11.4 | 10 |
pnCCD Gain |
Photon Energy (keV) |
Pulse Energy (mJ) |
Temperature Top/Bottom (degC) |
Run | Dark Run |
Proposal |
| 1/64 | 1.8 | 1.9 | -30 / -30 | 86 | 89 | 2720 |
pnCCD Gain |
Photon Energy (keV) |
Pulse Energy (mJ) |
Temperature Top/Bottom (degC) |
Run | Atten | Trans | Filters (microns) |
| 1/16 | 1.8 | 1.98 | -30 / -30 | 76 | 0.006 | 50 | 20 and 10 |
| 1/16 | 1.8 | 1.9 | -30 / -30 | 77 | 1e-5 | 1 | 20 and 10 |
| 1/16 | 1.8 | 1.9 | -30 / -30 | 79 | 11 | 0.47 | 5 and 10 |
The runs mentioned in Table 5.1 are not analyzed becasue full well capacity depends on the bias voltages applied to the detector, and the operating voltages of the top and bottom pnCCD sensors during these runs are not used anymore. Run 86 with gain 1/64 is also not analyzed because it is only one run at one intensity. We are looking at multiple runs with multiple intensities to derive the full well capacity. The three runs with pnCCD gain of 1/16 are analyzed. However, the FEL beam intensity for these runs was too low such that none of the pixels seem to have reached the full well capacity during these runs. Therefore, the analysis is not shown here.
Analyzing the Runs with Gain 1/256:
The data of runs 164, 166, 167, 168 and 169 were background subtracted (based on the dark run 170) and common mode corrected. Fig. 5.2 shows the corrected data for run 166 obtained by shining the highest intensity FEL beam onto a membrane. A lot of pixels in this image show possibility of saturation. The maximum intensity (in ADU) in this image belongs to the pixel with coordinates (row, column) \(=\) (476, 508). Fig. 5.3 shows the background subtracted signal (in ADU) in every image frame from run 166.
We then looked at a few individual pixels and plotted the PTC for these pixels. To obtain these plots, we have calculated the background subtracted and common mode corrected signal per pixel averaged over every 100 image frames. This is done for individual runs 164, 166, 167, 168 and 169. A final array is constructed by concatenating all individual mean signal arrays. The individual arrays are concatenated based on increasing level of FEL intensity. Similarly, the noise is also calculated as the standard deviation per pixel over every 100 image frames for individual runs 164, 166, 167, 168 and 169. A final array is constructed by concatenating all individual mean noise arrays corresponding to the run with the lowest FEL beam intensity to the one with the highest intensity. From the final arrays of mean signal and mean noise, we have constructed the PTCs, which are shown for a few selected pixels in Fig. 5.4 to Fig. 5.7.
To obtain the full well capacity and conversion gain, we have divided the data shown on Fig. 5.4 to 3 sections and have fit each section with a line using linear regression fitting method. The fits are shown in Fig. 5.5. The first fit has a slope of 0.7 and the second fit has the slope of 1.4. Therefore, these two sections represent the shot noise and fixed pattern noise (see Fig. 5.1), respectively. We do not seem to have FEL beam intensity low enough for this pixel to be able to observe the read noise section. The conversion gain is calculated from the shot noise section. An intersection between the linear fit on the shot noise section of the PTC and the x-axis gives \(\log(G)\), where \(G\) is the conversion gain in \(e^{-}/ADU\). Therefore, we obtain 1.32 \(e^{-}/ADU\) for the conversion gain using the PTC of the pixel with coordinates of (row,column) \(=\) (476,508).
The full well capacity is where the third fit crosses the x-axis, which is 4.498 on a logarithmic scale. Therefore, the full well capacity is \(10^{4.498}\), which is 31461 ADU. We can then use the conversion factor we obtained in \(e^{-}/ADU\) and convert the full well capacity from ADU to number of electrons. Doing so, we arrive at a full well capacity of 41529 \(e^{-}\).
Warning
- The measurement from which full well capacity can be determined must satisfy the conditions of a flat field measurement. In other words, the detector shall be shined on with a flat field FEL beam whose intensity is slowly increased until the saturation is reached. This was not the case for our measurement as can be seen in Fig. 5.2. We had a FEL beam very localized around the center of the CCD and everywhere else was rather dark. Therefore, not all pixels have reached full well capacity, and for this reason, we cannot calculate the full well capacity accross the detector consistently.
The offset and common mode corrected data from run 166. The image shows the highest intensity (in ADU) recorded in image frame 148. Gain was set to 1/256 and the pnCCD was operating in normal mode.
The offset and common mode corrected signal (in ADU) for every image frame from run 166 in a pixel located in row 476 and column 508. This pixel shows the highest intensity (in ADU) recorded during this run in image frame 148. Gain was set to 1/256 and the pnCCD was operating in normal mode.
The offset and common mode corrected signal averaged over every 100 image frames is calculated at the pixel with coordinates of (row,column) \(=\) (476,508) for individual runs 164, 166, 167, 168 and 169. The individual arrays are concatenated based on increasing level of FEL intensity. The natural logarithm of this final array is shown on the x-axis. The y-axis shows natural logarithm of the mean noise, where the latter is also calculated the same way as mean signal. The gain was set to 1/256 and pnCCD was operating in normal mode. A comparison of this figure and Fig. 5.1, shows that this particular pixel reaches full well capacity after which the pixel is saturated.
The fits of different sections on the PTC plot for a pixel with coordinates of (row,column) \(=\) (476,508). pnCCD gain was set to 1/256 and it was operating in normal mode. The shot noise and fixed pattern noise (see Fig. 5.1) are shown by the first and second fits, respectively. The first and third fits are used to obtain conversion gain(in \(e^{-}/ADU\)) and full well capacity (in ADU), respectively. These values are calculated to be 1.32 \(e^{-}/ADU\) and 31461 ADU, respectively. Using the conversion gain in \(e^{-}/ADU\), we converted the full well capacity in ADU to that in units of electrons and the result is: full well capacity \(=\) 41529 \(e^{-}\).
The PTC plot for a pixel with coordinates of (row,column) \(=\) (548,525). pnCCD gain was set to 1/256 and it was operating in normal mode. This pixel is not saturated because of its location with respect to the area where the FEL beam was shining on the detector. The PTC plot shows the shot noise and part of fixed pattern noise sections. Because this pixel is not saturated, we do not see the full well turning point on this plot. If this plot is fitted by two lines using the linear regression method, the slopes of the lines are 0.531 and 1.7. The former is rather close to the expected slope of 1/2, while the latter is bigger than the expected slope of one for the fixed pattern noise.
The PTC plot for a pixel with coordinates of (row,column) \(=\) (425,425). pnCCD gain was set to 1/256 and it was operating in normal mode. This pixel is far from being saturated because of its location with respect to the area where the FEL beam was shining on the detector. The PTC plot only shows the shot noise section because this pixel is far from saturation. If this plot is fitted by a line using the linear regression method, the slope of the line is 0.466, which is rather close to the expected slope of 1/2 for the shot noise section.