:sparkles: Developing PerturbationMatrix class by christinahedges · Pull Request #59 · SSDataLab/psfmachine

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

christinahedges · 2022-03-30T04:23:04Z

This PR is for developing a new PerturbationMatrix class which will ultimately factor our a lot of the headaches we're having inside machine

To Do

Documentation (docstrings!)
Merge into machine.py
Write some tests for Kepler/K2/TESS data

christinahedges · 2022-04-02T00:07:23Z

Roughly how to use:

If flux and flux_err are arrays with dimensions (ntimes x npixels)

P=PerturbationMatrix3d(time=time, dx=dx, dy=dy)
P.fit(flux, flux_err)
model=P.model()

model will then also have (ntimes x npixels)

jorgemarpa

This is great! I think the basic functionalities are implemented here.

Something that will be good to have is a pixel_mask argument in PerturbationMatrix3D so we can create matrix with the source_mask (for fitting) pixels and the uncontaminated_source_mask (for building) in one perturbation object.

I did a couple of "performance" test with TPF data and turned out that the self.model method runs really slow, about 1.7 sec/iter. We need to find why this is so slow.

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges · 2022-04-11T22:44:42Z

I added a PCA method so you can do something like:

y= [whitened_timeseries, whitened_timeseries]
p=PerturbationMatrix(...)
p.pca(y, ncomponents=10)
p.fit(y)

jorgemarpa

@christinahedges I found the bug with the inconsistent shape when doing self.model(). The problem is when using PerturbationMatrix through PerturbationMatrix3D that some attributes from the latter are not updated to reflect the addition of PCA components in self.vectors. See my comments for more details.

christinahedges · 2022-04-12T06:02:52Z

Any vectors that aren't the time polynomial are now set to zero mean in each "segment" during _clean_vectors method. The pca method can now be repeated and it will just update the pca components, not append ad infinitum

Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

jorgemarpa

This looks great! I tested a handful of component combinations and other parameters to check the matrices look Ok.
It fulfills what we always thought about the perturbation matrix API.

I just added a couple of comments regarding docstrings. Besides those, I think this is ready to be merged.

jorgemarpa · 2022-04-16T00:37:13Z

+
+ return func
+
+ def pca(self, y, ncomponents=5, smooth_time_scale=0):


to be consistent with the rest
Suggested change
defpca(self, y, ncomponents=5, smooth_time_scale=0):
defpca(self, y, ncomponents=3, smooth_time_scale=0):

jorgemarpa · 2022-04-16T00:37:48Z

+ Will add two time scales of principal components, definied by `long_time_scale`
+ and `med_time_scale`.


Suggested change
Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
and`med_time_scale`.

jorgemarpa · 2022-04-16T00:39:17Z

+ long_time_scale: float
+ The time scale where variability is considered "long" term. Should be
+ in the same units as `self.time`.
+ med_time_scale: float
+ The time scale where variability is considered "medium" term. Should be
+ in the same units as `self.time`. Variability longer than `long_time_scale`,
+ or shorter than `med_time_scale`, will be removed before building components


What is the scale of smooth_time_scale?
Suggested change
long_time_scale: float
Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
inthesameunitsas`self.time`.
med_time_scale: float
Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
smooth_time_scale: float
Amounttosmooththecomponents, usingasplineintime.
If0, thecomponentswillnotbesmoothed.

jorgemarpa · 2022-04-16T01:20:40Z

+ # def _clean_vectors(self):
+ # """Remove time polynomial from other vectors"""
+ # nvec = self.poly_order + 1
+ # if self.focus:
+ # nvec += 1
+ # if self.segments:
+ # s = nvec * (len(self.breaks) + 1)
+ # else:
+ # s = nvec
+ #
+ # if s != self.vectors.shape[1]:
+ # X = self.vectors[:, :s]
+ # w = np.linalg.solve(X.T.dot(X), X.T.dot(self.vectors[:, s:]))
+ # self.vectors[:, s:] -= X.dot(w)
+ # # Each segment has mean zero
+ # self.vectors[:, s:] -= np.asarray(
+ # [v[v != 0].mean() * (v != 0) for v in self.vectors[:, s:].T]
+ # ).T
+ # return


maybe we can still keep this function in case we want to remove the temporal trend for other components. But it isn't necessary with the current API.

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed

✨ Developing PerturbationMatrix class
05f4e26

christinahedges added the enhancement New feature or request label Mar 30, 2022

christinahedges added 3 commits March 29, 2022 21:29

✅ fixing flake8
172fbda

✨ added a 3D matrix
fb31d8b

🎨 fairly good sketch
d37a540

christinahedges requested a review from jorgemarpa April 4, 2022 20:02

jorgemarpa reviewed Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges commented Apr 5, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedgesand others added 6 commits April 5, 2022 11:45

Update src/psfmachine/perturbation.py
884f24d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

Update src/psfmachine/perturbation.py
18cca59

fixing identation and _clean_vectors when segments=True
3df9bea

⚡ faster!
b64b9cc

🎨 adding pixel masking
2d55aa1

📝 added docstrings
7310ed3

jorgemarpa mentioned this pull request Apr 6, 2022
Machine uses perturbation class #60
Merged
5 tasks

christinahedges changed the title ~~[WIP] ✨ Developing PerturbationMatrix class~~✨ Developing PerturbationMatrix classApr 8, 2022

christinahedges added the ready-to-merge label Apr 8, 2022

✨ Added PCA method
84a064c

christinahedges added 2 commits April 11, 2022 16:05

🎨 added fbpca explicitly as dependency
3da1557

📝 docs
1520001

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated
Comment threadsrc/psfmachine/perturbation.py Outdated

christinahedges added 2 commits April 11, 2022 22:48

🐛 fixing pca
78633dc

🐛 fix@jorgemarpabug
14babe3

📝 update docstrings
9d4a80f

jorgemarpa reviewed Apr 12, 2022
View reviewed changes

Comment threadsrc/psfmachine/perturbation.py Outdated

Update src/psfmachine/perturbation.py
556317d
Co-authored-by: Jorge Martínez-Palomera <jorgemarpa@users.noreply.github.com>

christinahedges added 8 commits April 12, 2022 13:12

✨ now pca has smoothed components
e97d3f5

🐛 update pca
0cbde46

🎨 merge
a00d69c

🎨 flake8
18fdc55

🎨 no comment block
eb9ce3c

📝 docstrings
2efb963

🎨 updating sane defaults, adding smooth PCA components
eaa807f

🎨 sane defaults
e1bc28e

jorgemarpa reviewed Apr 16, 2022
View reviewed changes

christinahedges added 2 commits April 18, 2022 15:09

📝 documentation
34a7622

🔀 merging
ed228ba

christinahedges merged commit a88567e into masterApr 18, 2022

jorgemarpa mentioned this pull request Dec 13, 2022
PSFMachine perturbation matrix needs a refactor #55
Closed


		return func

		def pca(self, y, ncomponents=5, smooth_time_scale=0):

	defpca(self, y, ncomponents=5, smooth_time_scale=0):
	defpca(self, y, ncomponents=3, smooth_time_scale=0):

		Will add two time scales of principal components, definied by `long_time_scale`
		and `med_time_scale`.

	Willaddtwotimescalesofprincipalcomponents, definiedby`long_time_scale`
	and`med_time_scale`.

-long_time_scale: float
-Thetimescalewherevariabilityisconsidered"long"term. Shouldbe
-inthesameunitsas`self.time`.
-med_time_scale: float
-Thetimescalewherevariabilityisconsidered"medium"term. Shouldbe
-inthesameunitsas`self.time`. Variabilitylongerthan`long_time_scale`,
-orshorterthan`med_time_scale`, willberemovedbeforebuildingcomponents
+smooth_time_scale: float
+Amounttosmooththecomponents, usingasplineintime.
+If0, thecomponentswillnotbesmoothed.

Uh oh!

Conversation

christinahedges commented Mar 30, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To Do

Uh oh!

christinahedges commented Apr 2, 2022

Uh oh!

jorgemarpa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christinahedges commented Apr 11, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorgemarpa left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

christinahedges commented Apr 12, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

jorgemarpa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

christinahedges commented Mar 30, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To Do

Uh oh!

christinahedges commented Apr 2, 2022

Uh oh!

jorgemarpa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christinahedges commented Apr 11, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorgemarpa left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

christinahedges commented Mar 30, 2022•
edited
Loading

christinahedges commented Apr 11, 2022•
edited
Loading

jorgemarpa left a comment •
edited
Loading

christinahedges commented Apr 12, 2022•
edited
Loading

christinahedges commented Mar 30, 2022•
edited
Loading

christinahedges commented Apr 11, 2022•
edited
Loading

jorgemarpa left a comment •
edited
Loading

christinahedges commented Apr 12, 2022•
edited
Loading

christinahedges commented Mar 30, 2022•
edited
Loading

christinahedges commented Apr 11, 2022•
edited
Loading