Skip to content

Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vkarampudi merged 13 commits into from
Feb 11, 2026
Merged

Changes thats need to be included for 0.48.1 patch release #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
330ce64
Struct2Tensor 0.48.0 Release
tfx-copybara Apr 3, 2025
5291fc1
Build documentation with `mkdocs` (#38)
vkarampudi Feb 6, 2026
ce97e66
only internal visibility change, no external change
tfx-copybara May 6, 2025
2e6fc33
Automated Code Change
tfx-copybara Aug 19, 2025
2ba7c6d
Automated Code Change
tfx-copybara Sep 5, 2025
cb5b1fc
no-op
tfx-copybara Sep 19, 2025
3bd9d30
Added pytype None checks to calculate.py
tfx-copybara Dec 16, 2025
5b6ed63
Replace unicode escaped characters in ipynb files
tfx-copybara Jan 14, 2026
a408605
Fixes that needs to be available in the main branch in order to cherr…
vkarampudi Feb 5, 2026
1c35bc9
docs(release): Update RELEASE.md and version for 0.48.0
vkarampudi Feb 6, 2026
de17d44
Update Dockerfile to build with serving version 2.17.1
vkarampudi Feb 8, 2026
a7941ee
Build and stage dynamic op libraries before wheel creation (#51)
czgdp1807 Feb 9, 2026
1964368
Enable OSS Bazel builds by fixing dependencies and proto paths (#50)
czgdp1807 Feb 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions RELEASE.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A
21 changes: 19 additions & 2 deletions WORKSPACE
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
"https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
],
strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
patches = ["//third_party:tensorflow.patch"],
patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
name = "com_google_protobuf",
sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
strip_prefix = "protobuf-4.25.6",
urls = [
"https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down
17 changes: 17 additions & 0 deletions build_common.sh
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
"${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
# Explicitly build the dynamic library targets that are needed for the wheel.
# These are required by the stamp_wheel function.
bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

RUNFILES_DIR=$(pwd)
cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down
160 changes: 80 additions & 80 deletions examples/prensor_playground.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion struct2tensor/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
"path.py",
],
deps = [
"@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
"@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
"@com_google_protobuf//:protobuf_python",
],
)
Expand Down
56 changes: 28 additions & 28 deletions struct2tensor/benchmarks/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
name = "benchmark_cc_proto",
deps = [":benchmark_proto"],
)

cc_test(
name = "serialization_benchmark",
srcs = ["serialization_benchmark.cc"],
deps = [
":benchmark_cc_proto",
"//testing/base/public:gunit",
"@com_google_absl//absl/flags:flag",
"@com_google_absl//absl/flags:parse",
"@com_google_absl//absl/random",
"@com_google_absl//absl/random:distributions",
"@com_google_protobuf//:protobuf",
"@org_tensorflow//tensorflow/core:feature_util",
"@org_tensorflow//tensorflow/core:protos_all_cc",
],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
":benchmark_proto_py_pb2",
":struct2tensor_benchmark_util",
"//struct2tensor",
"//testing/pybase",
"//testing/pybase:parameterized",
"//third_party/py/cpuinfo",
"//third_party/py/psutil",
# Google-internal dependencies - not available in OSS
# "//testing/pybase",
# "//testing/pybase:parameterized",
# "//third_party/py/cpuinfo",
# "//third_party/py/psutil",
],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
name = "struct2tensor_benchmark_util",
srcs = ["struct2tensor_benchmark_util.py"],
deps = [
"//file/colossus/public:cns",
"//third_party/py/cpuinfo",
"//third_party/py/psutil",
# Google-internal dependencies - not available in OSS
# "//file/colossus/public:cns",
# "//third_party/py/cpuinfo",
# "//third_party/py/psutil",
"@absl_py//absl/flags",
"@absl_py//absl/testing:parameterized",
],
Expand Down
9 changes: 8 additions & 1 deletion struct2tensor/calculate.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

def _create_value_error(self) -> ValueError:
"""Creates a ValueError, assuming there should be one for this node."""
assert self.value is not None
return ValueError("Expression {} returned the wrong type:"
" expected: {}"
" actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
options: calculate_options.Options,
feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
) -> None:
"""Calculates the values of the expressions in the graph."""
for node in self.ordered_node_list:
source_values = [self._node[id(x)].value for x in node.sources]
source_values = [
self._node[id(x)].value
for x in node.sources
if self._node[id(x)] is not None
and self._node[id(x)].value is not None
]
side_info = feed_dict[node.expression] if feed_dict and (
node.expression in feed_dict) else None
node.calculate(source_values, options, side_info=side_info)
Expand Down
3 changes: 1 addition & 2 deletions struct2tensor/kernels/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
"@com_google_absl//absl/base:endian",
"@com_google_absl//absl/container:flat_hash_map",
"@com_google_absl//absl/container:inlined_vector",
"@com_google_absl//absl/container:inlined_vector_internal",
"@com_google_absl//absl/memory",
"@com_google_absl//absl/strings",
"@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
"@com_google_absl//absl/base:endian",
"@com_google_absl//absl/container:flat_hash_map",
"@com_google_absl//absl/container:inlined_vector",
"@com_google_absl//absl/container:inlined_vector_internal",
"@com_google_absl//absl/strings",
"@com_google_absl//absl/types:span",
],
Expand Down
73 changes: 39 additions & 34 deletions struct2tensor/kernels/parquet/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
deps = [
":parent_indices_builder_lib",
":parquet_reader_util_lib",
"@com_google_absl//absl/status",
"@com_google_absl//absl/strings",
"@arrow//:arrow",
"//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
],
)

cc_test(
name = "parent_indices_builder_test",
srcs = ["parent_indices_builder_test.cc"],
deps = [
":parent_indices_builder_lib",
"//testing/base/public:gunit_main",
"@org_tensorflow//tensorflow/core:lib",
"@org_tensorflow//tensorflow/core:test",
],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
name = "parquet_reader_test",
srcs = ["parquet_reader_test.cc"],
data = [
"//struct2tensor/testdata/parquet_testdata",
],
deps = [
":parquet_reader_lib",
"//testing/base/public:gunit_main",
"@org_tensorflow//tensorflow/core:framework",
],
)

cc_test(
name = "parquet_reader_util_test",
srcs = ["parquet_reader_util_test.cc"],
data = [
"//struct2tensor/testdata/parquet_testdata",
],
deps = [
":parquet_reader_util_lib",
"//testing/base/public:gunit_main",
],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )
3 changes: 2 additions & 1 deletion struct2tensor/kernels/parquet/parquet_reader.cc
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
column_index, def_levels, rep_levels, &cumulative_values);

++(*messages_read);
if (tensorflow::errors::IsOutOfRange(s)) {
if (absl::IsOutOfRange(s)) {
break;
}
TF_RETURN_IF_ERROR(s);
Expand Down
3 changes: 3 additions & 0 deletions struct2tensor/ops/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down
5 changes: 2 additions & 3 deletions struct2tensor/proto/BUILD
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
name = "query_metadata_proto",
srcs = ["query_metadata.proto"],
deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
name = "query_metadata_py_pb2",
srcs = ["query_metadata.proto"],
api_version = 2,
oss_deps = [
"@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
"@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
],
proto_library = "query_metadata_proto",
)
Loading
, 'i'); if (__m === '*' || __re.test(location.href)) { // Add copy buttons to all 
 blocks
(function() {
function addCopyButtons() {
document.querySelectorAll('pre code').forEach(function(codeBlock) {
if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;
codeBlock.parentElement.setAttribute('data-copy-added', 'true');
var btn = document.createElement('button');
btn.textContent = 'Copy';
btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';
btn.onmouseover = function() { this.style.opacity = '1'; };
btn.onmouseout = function() { this.style.opacity = '0.7'; };
btn.onclick = function() {
navigator.clipboard.writeText(codeBlock.textContent).then(function() {
btn.textContent = 'Copied!';
setTimeout(function() { btn.textContent = 'Copy'; }, 1500);
});
};
codeBlock.parentElement.style.position = 'relative';
codeBlock.parentElement.appendChild(btn);
});
}
addCopyButtons();
// Re-run on dynamic content
var observer = new MutationObserver(addCopyButtons);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Force GitHub README to respect dark mode
(function() {
var style = document.createElement('style');
style.textContent = '
.markdown-body {
color-scheme: dark light;
}
.markdown-body pre { background: #161b22 !important; }
.markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }
.markdown-body table th, .markdown-body table td { border-color: #30363d !important; }
.markdown-body img { background: #0d1117; }
.markdown-body blockquote { border-left-color: #8b949e; }
.markdown-body hr { border-color: #30363d; }
';
document.head.appendChild(style);
})();
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Highlight search terms from Google/DuckDuckGo/Bing referrer
(function() {
var ref = document.referrer;
var terms = [];
if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {
var url = new URL(ref);
var q = url.searchParams.get('q') || url.searchParams.get('p');
if (q) {
terms = q.split(/\s+/).filter(function(t) { return t.length > 2; });
}
}
if (terms.length === 0) return;
var style = document.createElement('style');
style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';
document.head.appendChild(style);
function highlight(node) {
if (node.nodeType === 3) { // text node
var text = node.textContent;
var found = false;
terms.forEach(function(term) {
var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\]\\]/g, '\\') + ')', 'gi');
if (regex.test(text)) {
found = true;
var frag = document.createDocumentFragment();
var parts = text.split(regex);
parts.forEach(function(part, i) {
if (i % 2 === 0) {
frag.appendChild(document.createTextNode(part));
} else {
var span = document.createElement('span');
span.className = 'userscript-highlight';
span.textContent = part;
frag.appendChild(span);
}
});
node.parentNode.replaceChild(frag, node);
}
});
} else if (node.nodeType === 1 && node.childNodes) { // element
var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];
if (!skipTags.includes(node.tagName)) {
Array.from(node.childNodes).forEach(highlight);
}
}
}
highlight(document.body);
// Re-highlight on dynamic content
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1 || node.nodeType === 3) highlight(node);
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Strip utm_, fbclid, gclid, etc. from all links on page
(function() {
var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',
'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',
'ref', 'ref_src', 'source', 'medium', 'campaign'];
function cleanUrl(url) {
try {
var u = new URL(url, window.location.origin);
var changed = false;
trackingParams.forEach(function(p) {
if (u.searchParams.has(p)) {
u.searchParams.delete(p);
changed = true;
}
});
return changed ? u.toString() : url;
} catch (e) {
return url;
}
}
function cleanLinks() {
document.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
cleanLinks();
var observer = new MutationObserver(function(mutations) {
mutations.forEach(function(m) {
m.addedNodes.forEach(function(node) {
if (node.nodeType === 1) {
if (node.tagName === 'A') cleanLinks();
node.querySelectorAll('a[href]').forEach(function(a) {
var clean = cleanUrl(a.href);
if (clean !== a.href) a.href = clean;
});
}
});
});
});
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Auto-enable theater mode on YouTube
(function() {
function tryTheater() {
var btn = document.querySelector('button[aria-label="Theater mode"], ytd-player #player button[title="Theater mode"]');
if (btn && !btn.classList.contains('activated')) {
btn.click();
}
}
// Try immediately
tryTheater();
// Try after navigation (SPA)
var lastUrl = location.href;
setInterval(function() {
if (location.href !== lastUrl) {
lastUrl = location.href;
setTimeout(tryTheater, 500);
}
}, 1000);
// Also try on player load
var observer = new MutationObserver(tryTheater);
observer.observe(document.body, { childList: true, subtree: true });
})();
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Remove or un-stick sticky/fixed headers that block content
(function() {
function unstick() {
document.querySelectorAll('header, nav, [role="banner"], .header, .navbar, .sticky, .fixed-top, [style*="position: fixed"], [style*="position:sticky"]').forEach(function(el) {
if (el.style.position === 'fixed' || el.style.position === 'sticky' || getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {
el.style.position = 'static';
el.style.top = 'auto';
el.style.zIndex = 'auto';
}
});
}
unstick();
var observer = new MutationObserver(unstick);
observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });
})();
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading

, 'i');
if (__m === '*' || __re.test(location.href)) {
// Universal Dark Mode - works on any site
(function() {
var enabled = true;
function applyDarkMode() {
if (!enabled) return;
// Create style element if it doesn't exist
var style = document.getElementById('universal-dark-mode-style');
if (!style) {
style = document.createElement('style');
style.id = 'universal-dark-mode-style';
document.head.appendChild(style);
}
// Dark mode CSS - inverts colors but preserves images/video
style.textContent = '
/* Invert everything except media */
html {
filter: invert(1) hue-rotate(180deg) !important;
background: #1a1a2e !important;
}
/* Restore images, videos, iframes, canvas */
img, video, iframe, canvas, svg, picture, [style*="background-image"] {
filter: invert(1) hue-rotate(180deg) !important;
}
/* Preserve specific elements that should not be inverted */
.no-dark-mode, .no-dark-mode *,
[data-theme="light"], [data-theme="light"],
.ace_editor, .ace_editor *,
.CodeMirror, .CodeMirror *,
.monaco-editor, .monaco-editor *,
.markdown-body pre, .markdown-body pre *,
.highlight, .highlight *,
pre code, pre code * {
filter: none !important;
}
/* Fix common UI elements */
.modal, .popup, .dropdown-menu, .tooltip, .popover {
filter: invert(1) hue-rotate(180deg) !important;
background: #2d2d44 !important;
border-color: #444 !important;
}
/* Scrollbars */
::-webkit-scrollbar { background: #1a1a2e !important; }
::-webkit-scrollbar-thumb { background: #444 !important; }
::-webkit-scrollbar-thumb:hover { background: #555 !important; }
/* Selection */
::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }
';
}
function removeDarkMode() {
var style = document.getElementById('universal-dark-mode-style');
if (style) style.remove();
}
// Toggle with Alt+Shift+D
document.addEventListener('keydown', function(e) {
if (e.altKey && e.shiftKey && e.key === 'D') {
e.preventDefault();
enabled = !enabled;
if (enabled) {
applyDarkMode();
console.log('[Universal Dark Mode] Enabled');
} else {
removeDarkMode();
console.log('[Universal Dark Mode] Disabled');
}
}
});
// Apply on load
applyDarkMode();
// Re-apply on dynamic content
var observer = new MutationObserver(function(mutations) {
if (enabled && !document.getElementById('universal-dark-mode-style')) {
applyDarkMode();
}
});
observer.observe(document.head, { childList: true });
console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');
})();
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();
Changes thats need to be included for 0.48.1 patch release by vkarampudi · Pull Request #52 · google/struct2tensor · GitHub
Skip to content
Changes thats need to be included for 0.48.1 patch release#52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

 Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.

Already on GitHub?
Sign in
to your account

 Merged

vkarampudi
merged 13 commits into

from

Feb 11, 2026
 Merged


Changes thats need to be included for 0.48.1 patch release
#52
Show file tree
Hide file tree
 Changes from all commits
Commits

Show all changes


13 commits


Select commit
Hold shift + click to select a range
330ce64

Struct2Tensor 0.48.0 Release


tfx-copybara Apr 3, 2025
5291fc1

Build documentation with `mkdocs` (#38)


vkarampudi Feb 6, 2026
ce97e66

only internal visibility change, no external change


tfx-copybara May 6, 2025
2e6fc33

Automated Code Change


tfx-copybara Aug 19, 2025
2ba7c6d

Automated Code Change


tfx-copybara Sep 5, 2025
cb5b1fc

no-op


tfx-copybara Sep 19, 2025
3bd9d30

Added pytype None checks to calculate.py


tfx-copybara Dec 16, 2025
5b6ed63

Replace unicode escaped characters in ipynb files


tfx-copybara Jan 14, 2026
a408605

Fixes that needs to be available in the main branch in order to cherr…


vkarampudi Feb 5, 2026
1c35bc9

docs(release): Update RELEASE.md and version for 0.48.0


vkarampudi Feb 6, 2026
de17d44

Update Dockerfile to build with serving version 2.17.1


vkarampudi Feb 8, 2026
a7941ee

Build and stage dynamic op libraries before wheel creation (#51)


czgdp1807 Feb 9, 2026
1964368

Enable OSS Bazel builds by fixing dependencies and proto paths (#50)


czgdp1807 Feb 9, 2026

File filter

Filter by extension
Filter by extension
Conversations

Failed to load comments. 
Loading
Jump to

Jump to file


Failed to load files. 
Loading
 Diff view

 Diff view


3 changes: 1 addition & 2 deletions
RELEASE.md

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,5 +16,4 @@

## Deprecations

* N/A

* N/A

21 changes: 19 additions & 2 deletions
WORKSPACE

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,8 @@ tf_configure(name = "local_config_tf")
# 3. Request the new archive to be mirrored on mirror.bazel.build for more
# reliable downloads.

_TENSORFLOW_GIT_COMMIT = "5bc9d26649cca274750ad3625bd93422617eed4b" # tf 2.16.1
_TENSORFLOW_ARCHIVE_SHA256 = "fe592915c85d1a89c20f3dd89db0772ee22a0fbda78e39aa46a778d638a96abc"
_TENSORFLOW_GIT_COMMIT = "3c92ac03cab816044f7b18a86eb86aa01a294d95" # tf 2.17.1
_TENSORFLOW_ARCHIVE_SHA256 = "317dd95c4830a408b14f3e802698eb68d70d81c7c7cfcd3d28b0ba023fe84a68"

http_archive(
 name = "org_tensorflow",
Expand All@@ -51,6 +51,8 @@ http_archive(
 "https://github.com/tensorflow/tensorflow/archive/%s.tar.gz" % _TENSORFLOW_GIT_COMMIT,
 ],
 strip_prefix = "tensorflow-%s" % _TENSORFLOW_GIT_COMMIT,
 patches = ["//third_party:tensorflow.patch"],
 patch_args = ["-p1"],
)

load("//third_party:python_configure.bzl", "local_python_configure")
Expand All@@ -62,6 +64,21 @@ local_python_configure(name = "local_execution_config_python")
load("//struct2tensor:workspace.bzl", "struct2tensor_workspace")
struct2tensor_workspace()

# ===== Protobuf 4.25.6 dependency =====
# Must be declared BEFORE TensorFlow's workspaces to override the version they pull
http_archive(
 name = "com_google_protobuf",
 sha256 = "4e6727bc5d23177edefa3ad86fd2f5a92cd324151636212fd1f7f13aef3fd2b7",
 strip_prefix = "protobuf-4.25.6",
 urls = [
 "https://github.com/protocolbuffers/protobuf/archive/v4.25.6.tar.gz",
 ],
)

# Load Protobuf dependencies
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()

# Initialize TensorFlow's external dependencies.
load("@org_tensorflow//tensorflow:workspace3.bzl", "tf_workspace3")
tf_workspace3()
Expand Down

17 changes: 17 additions & 0 deletions
build_common.sh

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -62,6 +62,21 @@ function install_tensorflow() {
 "${PYTHON_BIN_PATH}" -c 'import tensorflow as tf; print(tf.version.VERSION)'
}

function build_dynamic_libraries() {
 # Explicitly build the dynamic library targets that are needed for the wheel.
 # These are required by the stamp_wheel function.
 bazel build //struct2tensor/ops:_decode_proto_map_op.so || exit 1;
 bazel build //struct2tensor/ops:_decode_proto_sparse_op.so || exit 1;
 bazel build //struct2tensor/ops:_run_length_before_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_any_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_equi_join_indices_op.so || exit 1;
 bazel build //struct2tensor/ops:_parquet_dataset_op.so || exit 1;

 RUNFILES_DIR=$(pwd)
 cp -f bazel-bin/struct2tensor/ops/*.so ${RUNFILES_DIR}/struct2tensor/ops/

}

set -x

for i in "$@"; do
Expand DownExpand Up@@ -116,6 +131,8 @@ if [[ ("${TF_VERSION}" == "NIGHTLY_TF") || ("${TF_VERSION}" == "NIGHTLY_TF_2") ]

fi

build_dynamic_libraries

# :build_pip_package builds and links struct2tensor ops against a TF
# installation and packages the result dynamic libraries.
bazel run -c opt \
Expand Down

160 changes: 80 additions & 80 deletions
examples/prensor_playground.ipynb

View file


Open in desktop


Large diffs are not rendered by default.


3 changes: 2 additions & 1 deletion
struct2tensor/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -15,6 +15,7 @@
# Prensors provide a way to parse protocol buffers and other structured data in tensorflow
# simply and efficiently.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
# Placeholder: load py_test

load("//struct2tensor:struct2tensor.bzl", "s2t_pytype_library")
Expand DownExpand Up@@ -83,7 +84,7 @@ s2t_pytype_library(
 "path.py",
 ],
 deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 "@com_google_protobuf//:protobuf_python",
 ],
)
Expand Down

56 changes: 28 additions & 28 deletions
struct2tensor/benchmarks/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -12,12 +12,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.

# Placeholder: load py_proto_library
# Placeholder: load py_binary
# Placeholder: load py_library
# Placeholder: load py_test
load("//third_party/protobuf/bazel:proto_library.bzl", "proto_library")
load("//tools/build_defs/proto/cpp:cc_proto_library.bzl", "cc_proto_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "py_proto_library")

licenses(["notice"])

Expand All@@ -33,26 +29,28 @@ py_proto_library(
 deps = [":benchmark_proto"],
)

# Use native cc_proto_library instead of custom macro since we already have proto_library defined
cc_proto_library(
 name = "benchmark_cc_proto",
 deps = [":benchmark_proto"],
)

cc_test(
 name = "serialization_benchmark",
 srcs = ["serialization_benchmark.cc"],
 deps = [
 ":benchmark_cc_proto",
 "//testing/base/public:gunit",
 "@com_google_absl//absl/flags:flag",
 "@com_google_absl//absl/flags:parse",
 "@com_google_absl//absl/random",
 "@com_google_absl//absl/random:distributions",
 "@com_google_protobuf//:protobuf",
 "@org_tensorflow//tensorflow/core:feature_util",
 "@org_tensorflow//tensorflow/core:protos_all_cc",
 ],
)
# Google-internal benchmark - commented out for OSS
# cc_test(
# name = "serialization_benchmark",
# srcs = ["serialization_benchmark.cc"],
# deps = [
# ":benchmark_cc_proto",
# "//testing/base/public:gunit",
# "@com_google_absl//absl/flags:flag",
# "@com_google_absl//absl/flags:parse",
# "@com_google_absl//absl/random",
# "@com_google_absl//absl/random:distributions",
# "@com_google_protobuf//:protobuf",
# "@org_tensorflow//tensorflow/core:feature_util",
# "@org_tensorflow//tensorflow/core:protos_all_cc",
# ],
# )

py_library(
 name = "struct2tensor_benchmark_lib",
Expand All@@ -61,10 +59,11 @@ py_library(
 ":benchmark_proto_py_pb2",
 ":struct2tensor_benchmark_util",
 "//struct2tensor",
 "//testing/pybase",
 "//testing/pybase:parameterized",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//testing/pybase",
 # "//testing/pybase:parameterized",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 ],
)

Expand DownExpand Up@@ -110,9 +109,10 @@ py_library(
 name = "struct2tensor_benchmark_util",
 srcs = ["struct2tensor_benchmark_util.py"],
 deps = [
 "//file/colossus/public:cns",
 "//third_party/py/cpuinfo",
 "//third_party/py/psutil",
 # Google-internal dependencies - not available in OSS
 # "//file/colossus/public:cns",
 # "//third_party/py/cpuinfo",
 # "//third_party/py/psutil",
 "@absl_py//absl/flags",
 "@absl_py//absl/testing:parameterized",
 ],
Expand Down

9 changes: 8 additions & 1 deletion
struct2tensor/calculate.py

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -259,6 +259,7 @@ def __str__(self) -> str:

 def _create_value_error(self) -> ValueError:
 """Creates a ValueError, assuming there should be one for this node."""
 assert self.value is not None
 return ValueError("Expression {} returned the wrong type:"
 " expected: {}"
 " actual: {}.".format(
Expand DownExpand Up@@ -353,8 +354,14 @@ def calculate_values(
 options: calculate_options.Options,
 feed_dict: Optional[Dict[expression.Expression, prensor.Prensor]] = None
 ) -> None:
 """Calculates the values of the expressions in the graph."""
 for node in self.ordered_node_list:
 source_values = [self._node[id(x)].value for x in node.sources]
 source_values = [
 self._node[id(x)].value
 for x in node.sources
 if self._node[id(x)] is not None
 and self._node[id(x)].value is not None
 ]
 side_info = feed_dict[node.expression] if feed_dict and (
 node.expression in feed_dict) else None
 node.calculate(source_values, options, side_info=side_info)
Expand Down

3 changes: 1 addition & 2 deletions
struct2tensor/kernels/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,7 @@

# CPU kernels for struct2tensors.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -68,7 +69,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/memory",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
Expand DownExpand Up@@ -104,7 +104,6 @@ s2t_dynamic_library(
 "@com_google_absl//absl/base:endian",
 "@com_google_absl//absl/container:flat_hash_map",
 "@com_google_absl//absl/container:inlined_vector",
 "@com_google_absl//absl/container:inlined_vector_internal",
 "@com_google_absl//absl/strings",
 "@com_google_absl//absl/types:span",
 ],
Expand Down

73 changes: 39 additions & 34 deletions
struct2tensor/kernels/parquet/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -14,6 +14,8 @@
#
# Parquet Dataset Kernel Implementation.

load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_library")

package(
Expand DownExpand Up@@ -60,6 +62,7 @@ cc_library(
 deps = [
 ":parent_indices_builder_lib",
 ":parquet_reader_util_lib",
 "@com_google_absl//absl/status",
 "@com_google_absl//absl/strings",
 "@arrow//:arrow",
 "//struct2tensor/kernels:vector_to_tensor",
Expand DownExpand Up@@ -96,38 +99,40 @@ cc_library(
 ],
)

cc_test(
 name = "parent_indices_builder_test",
 srcs = ["parent_indices_builder_test.cc"],
 deps = [
 ":parent_indices_builder_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:lib",
 "@org_tensorflow//tensorflow/core:test",
 ],
)
# Test file parent_indices_builder_test.cc does not exist, commenting out
# cc_test(
# name = "parent_indices_builder_test",
# srcs = ["parent_indices_builder_test.cc"],
# deps = [
# ":parent_indices_builder_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:lib",
# "@org_tensorflow//tensorflow/core:test",
# ],
# )

cc_test(
 name = "parquet_reader_test",
 srcs = ["parquet_reader_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_lib",
 "//testing/base/public:gunit_main",
 "@org_tensorflow//tensorflow/core:framework",
 ],
)

cc_test(
 name = "parquet_reader_util_test",
 srcs = ["parquet_reader_util_test.cc"],
 data = [
 "//struct2tensor/testdata/parquet_testdata",
 ],
 deps = [
 ":parquet_reader_util_lib",
 "//testing/base/public:gunit_main",
 ],
)
# Test file parquet_reader_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_test",
# srcs = ["parquet_reader_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# "@org_tensorflow//tensorflow/core:framework",
# ],
# )
# Test file parquet_reader_util_test.cc does not exist, commenting out
# cc_test(
# name = "parquet_reader_util_test",
# srcs = ["parquet_reader_util_test.cc"],
# data = [
# "//struct2tensor/testdata/parquet_testdata",
# ],
# deps = [
# ":parquet_reader_util_lib",
# "@com_google_googletest//:gtest_main", # Replaces Google-internal //testing/base/public:gunit_main
# ],
# )

3 changes: 2 additions & 1 deletion
struct2tensor/kernels/parquet/parquet_reader.cc

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -16,6 +16,7 @@ limitations under the License.

#include <memory>

#include "absl/status/status.h"
#include "absl/strings/str_cat.h"
#include "struct2tensor/kernels/parquet/parquet_reader_util.h"
#include "struct2tensor/kernels/vector_to_tensor.h"
Expand DownExpand Up@@ -440,7 +441,7 @@ tensorflow::Status ParquetReader::ReadOneColumnTemplated(
 column_index, def_levels, rep_levels, &cumulative_values);

 ++(*messages_read);
 if (tensorflow::errors::IsOutOfRange(s)) {
 if (absl::IsOutOfRange(s)) {
 break;
 }
 TF_RETURN_IF_ERROR(s);
Expand Down

3 changes: 3 additions & 0 deletions
struct2tensor/ops/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
load("@rules_cc//cc:cc_library.bzl", "cc_library")
load("@rules_cc//cc:cc_test.bzl", "cc_test")

# Placeholder: load py_test
load("//struct2tensor:struct2tensor.bzl", "s2t_dynamic_binary", "s2t_dynamic_library", "s2t_gen_op_wrapper_py", "s2t_pytype_library")

Expand Down

5 changes: 2 additions & 3 deletions
struct2tensor/proto/BUILD

View file


Open in desktop

Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ licenses(["notice"])
s2t_proto_library(
 name = "query_metadata_proto",
 srcs = ["query_metadata.proto"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:cc_metadata_v0_proto_cc"],
 deps = ["@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto"],
)

s2t_proto_library_cc(
Expand All@@ -19,10 +19,9 @@ s2t_proto_library_cc(

s2t_proto_library_py(
 name = "query_metadata_py_pb2",
 srcs = ["query_metadata.proto"],
 api_version = 2,
 oss_deps = [
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:py_metadata_v0_proto_py",
 "@com_github_tensorflow_metadata//tensorflow_metadata/proto/v0:metadata_v0_proto_py_pb2",
 ],
 proto_library = "query_metadata_proto",
)
Loading