Energy calibration bias shift

[rettigl]

Shift energy calibration by bias voltages, and apply shift of dataset bias upon adding energy column.
Unify dataframe channels config

closes#399
closes#138

[zain-sohail]

I understand that you want to merge to v1 feature branch because these are all breaking changes?

[rettigl]

I understand that you want to merge to v1 feature branch because these are all breaking changes?

Yes, they reqire configuration changes

[coveralls]

Pull Request Test Coverage Report for Build 9553353548

Details

• 76 of 87(87.36%) changed or added relevant lines in 3 files are covered.
• 3 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.01%) to 91.952%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	51	60	85.0%

Files with Coverage Reduction	New Missed Lines	%
sed/binning/numba_bin.py	3	87.74%

Totals
Change from base Build 9522245431:	-0.01%
Covered Lines:	6490
Relevant Lines:	7058

---

💛 - Coveralls

[rettigl]

This code here unfortunately performs rather poorly. The generation of the per-file data columns alone make the computation ~20% or so slower, adding the energy offset per electron multiplies the binning time by a factor of almost 4!

[coveralls]

Pull Request Test Coverage Report for Build 9587172660

Details

• 81 of 99(81.82%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.05%) to 92.009%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	17	18	94.44%
sed/core/processor.py	6	9	66.67%
sed/loader/mpes/loader.py	58	72	80.56%

Totals
Change from base Build 9522245431:	0.05%
Covered Lines:	6494
Relevant Lines:	7058

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9587913234

Details

• 87 of 99(87.88%) changed or added relevant lines in 3 files are covered.
• 3 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.01%) to 91.952%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	62	72	86.11%

Files with Coverage Reduction	New Missed Lines	%
sed/binning/numba_bin.py	3	87.74%

Totals
Change from base Build 9522245431:	-0.01%
Covered Lines:	6490
Relevant Lines:	7058

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9588080384

Details

• 87 of 99(87.88%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.08%) to 91.995%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	62	72	86.11%

Totals
Change from base Build 9588067031:	0.08%
Covered Lines:	6493
Relevant Lines:	7058

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9628853930

Details

• 87 of 99(87.88%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.08%) to 91.995%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	62	72	86.11%

Totals
Change from base Build 9588067031:	0.08%
Covered Lines:	6493
Relevant Lines:	7058

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9628937920

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.04%) to 91.956%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9588067031:	0.04%
Covered Lines:	6493
Relevant Lines:	7061

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9635909706

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.04%) to 91.894%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9635884851:	0.04%
Covered Lines:	6462
Relevant Lines:	7032

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9635950194

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• 3 unchanged lines in 1 file lost coverage.
• Overall coverage increased (+0.02%) to 91.88%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Files with Coverage Reduction	New Missed Lines	%
sed/binning/numba_bin.py	3	87.5%

Totals
Change from base Build 9635919510:	0.02%
Covered Lines:	6461
Relevant Lines:	7032

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9636169386

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.04%) to 91.894%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9635919510:	0.04%
Covered Lines:	6462
Relevant Lines:	7032

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9636247350

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.08%) to 91.894%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9636221019:	0.08%
Covered Lines:	6462
Relevant Lines:	7032

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9652315474

Details

• 104 of 116(89.66%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.04%) to 91.897%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/core/processor.py	7	9	77.78%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9652279283:	0.04%
Covered Lines:	6464
Relevant Lines:	7034

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9659018433

Details

• 104 of 116(89.66%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.1%) to 91.914%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	25	27	92.59%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9652390026:	0.1%
Covered Lines:	6468
Relevant Lines:	7037

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9733951548

Details

• 104 of 116(89.66%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.05%) to 92.511%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	25	27	92.59%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9733946272:	0.05%
Covered Lines:	6905
Relevant Lines:	7464

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9757007395

Details

• 104 of 116(89.66%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.05%) to 92.511%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	25	27	92.59%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9756916343:	0.05%
Covered Lines:	6905
Relevant Lines:	7464

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9759204395

Details

• 104 of 116(89.66%) changed or added relevant lines in 3 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.05%) to 92.511%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	25	27	92.59%
sed/loader/mpes/loader.py	70	80	87.5%

Totals
Change from base Build 9756916343:	0.05%
Covered Lines:	6905
Relevant Lines:	7464

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 9847465611

Details

• 159 of 169(94.08%) changed or added relevant lines in 5 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.1%) to 92.599%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
sed/calibrator/energy.py	25	27	92.59%
sed/loader/mpes/loader.py	80	88	90.91%

Totals
Change from base Build 9845601339:	0.1%
Covered Lines:	6969
Relevant Lines:	7526

---

💛 - Coveralls

[zain-sohail]

I have tried to review but currently, I don't understand the internals of the calibration methods enough to review this properly. I have tried because I wanted to adapt these for the side band correction but failed to understand.
Honestly, It'd be great if a small guide on the internals can be made. We can then also put it in user documentation.
If that's too much, if you could explain why you made these changes, I should be able to review based on that.

The only thing I noticed currently was that the tutorials still have the ref_id (in methods find_bias_peaks etc.) whereas the code only uses the ref_energy now.

[rettigl]

I have tried to review but currently, I don't understand the internals of the calibration methods enough to review this properly. I have tried because I wanted to adapt these for the side band correction but failed to understand. Honestly, It'd be great if a small guide on the internals can be made. We can then also put it in user documentation. If that's too much, if you could explain why you made these changes, I should be able to review based on that.

The only thing I noticed currently was that the tutorials still have the ref_id (in methods find_bias_peaks etc.) whereas the code only uses the ref_energy now.

Thank you for your effort. I will for now comment on the changes to explain them better. If I find time, I can also make a more general documentation item out of this.

[rettigl]

For using the calibration function for features from sidebands, do the following:

1. Extract the TOF values of the sidebands (e.g. TOF=[t0, t1, t2, t3, t4, t5, t6]).
2. Generate the corresponding energies, shifted by the bias voltage of your measurement, e.g. energies=[-3hv, -2hv, -hv, 0, hv, 2hv, 3hv] - (maybe +) bias
3. Calculate the energy of the first side band, e.g. E_corelevel - 3hv
4. Provide these values to ec.calibrate(ref_energy=reference, landmarks=TOF, biases=energies, ...)
   This should work, I think.

[zain-sohail]

Just a few minor comments. Thanks for explaining! Helped a lot. Could be a start to the calibration guide later.