Uh oh!
There was an error while loading. Please reload this page.
[SPARK-25345][ML] Deprecate public APIs from ImageSchema - #22349
[SPARK-25345][ML] Deprecate public APIs from ImageSchema#22349WeichenXu123 wants to merge 5 commits into
Conversation
SparkQA
commented
Sep 6, 2018
Test build #95749 has finished for PR 22349 at commit
|
SparkQA
commented
Sep 6, 2018
Test build #95751 has finished for PR 22349 at commit
|
| An attribute of this module that contains the instance of :class:`_ImageSchema`. | ||
| .. note:: Deprecated in 2.4.0. Use `spark.read.format("image").load(path)` instead and |
There was a problem hiding this comment.
FYI, that datasource wouldn't be able to replace toNDArray and toImage which require numpy. Might be better leave a note that it needs manual conversion with NumPy API for both.
| */ | ||
| @Experimental | ||
| @Since("2.3.0") | ||
| @deprecated("use `spark.read.format(\"image\").load(path)` and this `ImageSchema` will be " + |
There was a problem hiding this comment.
There are other methods defined under ImageSchema that are not covered by the image data source. So we shall only deprecate readImages and leave other public methods as experimental. Same for Python.
SparkQA
commented
Sep 7, 2018
Test build #95796 has finished for PR 22349 at commit
|
| .. note:: If sample ratio is less than 1, sampling uses a PathFilter that is efficient but | ||
| potentially non-deterministic. | ||
| .. note:: Deprecated in 2.4.0. Use `spark.read.format("image").load(path)` instead and |
There was a problem hiding this comment.
Would it make sense to issue a warning when people call this code path (e.g. warnings.warn)? We do it in some of the other Python deprecated APIs but not all.
SparkQA
commented
Sep 8, 2018
Test build #95817 has finished for PR 22349 at commit
|
| from pyspark import SparkContext | ||
| from pyspark.sql.types import Row, _create_row, _parse_datatype_json_string | ||
| from pyspark.sql import DataFrame, SparkSession | ||
| import warnings |
There was a problem hiding this comment.
Technically builtin package should be ordered above per PEP 8. I wonder why this is not caught.
import sys
import warnings
import numpy as np
from pyspark import SparkContext
from pyspark.sql.types import Row, _create_row, _parse_datatype_json_string
from pyspark.sql import DataFrame, SparkSession
| """ | ||
| warnings.warn("`ImageSchema.readImage` is deprecated. " + | ||
| "Use `spark.read.format(\"image\").load(path)` instead.") |
There was a problem hiding this comment.
warnings.warn(..., DeprecationWarning)
mengxr
commented
Sep 8, 2018
@WeichenXu123 Could you address the comments? |
SparkQA
commented
Sep 8, 2018
Test build #95831 has finished for PR 22349 at commit
|
## What changes were proposed in this pull request? Deprecate public APIs from ImageSchema. ## How was this patch tested? N/A Closes#22349 from WeichenXu123/image_api_deprecate. Authored-by: WeichenXu <weichen.xu@databricks.com> Signed-off-by: Xiangrui Meng <meng@databricks.com> (cherry picked from commit 08c02e6) Signed-off-by: Xiangrui Meng <meng@databricks.com>
mengxr
commented
Sep 8, 2018
LGTM. Merged into master and branch-2.4. Thanks! |
What changes were proposed in this pull request?
Deprecate public APIs from ImageSchema.
How was this patch tested?
N/A