Skip to content

Commit 7f94cc8

Browse files
timofey-stepanovcopybara-github
timofey-stepanov
authored and
copybara-github
committed
Move flatten_cyclic_references to contrib
The primary goal is to set up a skeleton for the C++ part of contrib package. But the operator does not seem to belong to kd.core — so a good target to practice. PiperOrigin-RevId: 891875991 Change-Id: Ie68ddea851c263030da852c1b12047e1c130f915
1 parent 5226695 commit 7f94cc8

23 files changed

Lines changed: 310 additions & 201 deletions

‎docs/api_reference.md‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -756,24 +756,6 @@ Args:
756756
Returns:
757757
A new immutable DataBag with only the reachable attrs from &#39;ds&#39;.</code></pre>
758758

759-
### `kd.core.flatten_cyclic_references(x, *, max_recursion_depth)` {#kd.core.flatten_cyclic_references}
760-
761-
<pre class="no-copy"><code class="lang-text no-auto-prettify">Creates a DataSlice with tree-like copy of the input DataSlice.
762-
763-
The entities themselves and all their attributes including both top-level and
764-
non-top-level attributes are cloned (with new ItemIds) while creating the
765-
tree-like copy. The max_recursion_depth argument controls the maximum number
766-
of times the same entity can occur on the path from the root to a leaf.
767-
Note: resulting DataBag might have an exponential size, compared to the input
768-
DataBag.
769-
770-
Args:
771-
x: DataSlice to flatten.
772-
max_recursion_depth: Maximum recursion depth.
773-
774-
Returns:
775-
A DataSlice with tree-like attributes structure.</code></pre>
776-
777759
### `kd.core.follow(x)` {#kd.core.follow}
778760
Aliases:
779761

@@ -11803,6 +11785,24 @@ External contributions not necessarily endorsed by Koda.
1180311785

1180411786
<pre class="no-copy"><code class="lang-text no-auto-prettify">Computes average rank natively in koladata.</code></pre>
1180511787

11788+
### `kd_ext.contrib.flatten_cyclic_references(x, *, max_recursion_depth)` {#kd_ext.contrib.flatten_cyclic_references}
11789+
11790+
<pre class="no-copy"><code class="lang-text no-auto-prettify">Creates a DataSlice with tree-like copy of the input DataSlice.
11791+
11792+
The entities themselves and all their attributes including both top-level and
11793+
non-top-level attributes are cloned (with new ItemIds) while creating the
11794+
tree-like copy. The max_recursion_depth argument controls the maximum number
11795+
of times the same entity can occur on the path from the root to a leaf.
11796+
Note: resulting DataBag might have an exponential size, compared to the input
11797+
DataBag.
11798+
11799+
Args:
11800+
x: DataSlice to flatten.
11801+
max_recursion_depth: Maximum recursion depth.
11802+
11803+
Returns:
11804+
A DataSlice with tree-like attributes structure.</code></pre>
11805+
1180611806
### `kd_ext.contrib.pearson_correlation(x: DataSlice, y: DataSlice) -> DataSlice` {#kd_ext.contrib.pearson_correlation}
1180711807

1180811808
<pre class="no-copy"><code class="lang-text no-auto-prettify">Computes Pearson correlation for koladata slices x and y.</code></pre>

‎koladata/BUILD‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ package_group(
4343
package_group(
4444
name = "experimental_users",
4545
packages = [
46+
"//py/koladata/ext/...",
4647
],
4748
)
4849

‎koladata/contrib/BUILD‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# C++-only parts of Koda Contrib. See the rest in //py/koladata/ext/contrib.
16+
17+
load("@rules_cc//cc:cc_library.bzl", "cc_library")
18+
19+
package(default_visibility = ["//visibility:private"])
20+
21+
licenses(["notice"])
22+
23+
cc_library(
24+
name = "operators",
25+
srcs = ["operators.cc"],
26+
visibility = ["//py/koladata/ext/contrib:__pkg__"],
27+
deps = [
28+
":flatten_cyclic_references",
29+
"//koladata:data_slice_qtype",
30+
"//koladata/internal/op_utils:qexpr",
31+
],
32+
alwayslink = 1,
33+
)
34+
35+
cc_library(
36+
name = "flatten_cyclic_references",
37+
srcs = ["flatten_cyclic_references.cc"],
38+
hdrs = ["flatten_cyclic_references.h"],
39+
deps = [
40+
"//koladata:data_bag",
41+
"//koladata:data_slice",
42+
"//koladata:data_slice_qtype",
43+
"//koladata/internal:data_bag",
44+
"//koladata/internal:data_item",
45+
"//koladata/internal:data_slice",
46+
"//koladata/internal:dtype",
47+
"//koladata/internal:non_deterministic_token",
48+
"//koladata/internal:object_id",
49+
"//koladata/internal:schema_attrs",
50+
"//koladata/internal/op_utils:traverse_helper",
51+
"@com_google_absl//absl/container:flat_hash_map",
52+
"@com_google_absl//absl/hash",
53+
"@com_google_absl//absl/log",
54+
"@com_google_absl//absl/log:check",
55+
"@com_google_absl//absl/status",
56+
"@com_google_absl//absl/status:statusor",
57+
"@com_google_absl//absl/strings",
58+
"@com_google_absl//absl/strings:string_view",
59+
"@com_google_arolla//arolla/dense_array/qtype",
60+
"@com_google_arolla//arolla/jagged_shape/dense_array/util",
61+
"@com_google_arolla//arolla/qtype",
62+
"@com_google_arolla//arolla/util",
63+
"@com_google_arolla//arolla/util:status_backport",
64+
],
65+
)

koladata/operators/flatten_cyclic_references.cc renamed to koladata/contrib/flatten_cyclic_references.cc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
// See the License for the specific language governing permissions and
1313
// limitations under the License.
1414
//
15-
#include "koladata/operators/flatten_cyclic_references.h"
15+
#include "koladata/contrib/flatten_cyclic_references.h"
1616

1717
#include <algorithm>
1818
#include <cstddef>
@@ -48,7 +48,7 @@
4848
#include "koladata/internal/slice_builder.h"
4949
#include "arolla/util/status_macros_backport.h"
5050

51-
namespace koladata::ops {
51+
namespace koladata::contrib {
5252

5353
namespace {
5454

@@ -326,4 +326,4 @@ absl::StatusOr<DataSlice> FlattenCyclicReferences(
326326
});
327327
}
328328

329-
} // namespace koladata::ops
329+
} // namespace koladata::contrib

koladata/operators/flatten_cyclic_references.h renamed to koladata/contrib/flatten_cyclic_references.h

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,22 +12,22 @@
1212
// See the License for the specific language governing permissions and
1313
// limitations under the License.
1414
//
15-
#ifndef KOLADATA_OPERATORS_TRANSFORM_H_
16-
#define KOLADATA_OPERATORS_TRANSFORM_H_
15+
#ifndef KOLADATA_CONTRIB_FLATTEN_CYCLIC_REFERENCES_H_
16+
#define KOLADATA_CONTRIB_FLATTEN_CYCLIC_REFERENCES_H_
1717

1818
#include <cstdint>
1919

2020
#include "absl/status/statusor.h"
2121
#include "koladata/data_slice.h"
2222
#include "koladata/internal/non_deterministic_token.h"
2323

24-
namespace koladata::ops {
24+
namespace koladata::contrib {
2525

26-
// kd.core.flatten_cyclic_references
26+
// kd_ext.contrib.flatten_cyclic_references
2727
absl::StatusOr<DataSlice> FlattenCyclicReferences(
2828
const DataSlice& ds, int64_t max_recursion_depth,
2929
internal::NonDeterministicToken = {});
3030

31-
} // namespace koladata::ops
31+
} // namespace koladata::contrib
3232

33-
#endif // KOLADATA_OPERATORS_TRANSFORM_H_
33+
#endif // KOLADATA_CONTRIB_FLATTEN_CYCLIC_REFERENCES_H_

‎koladata/contrib/operators.cc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
// Copyright 2025 Google LLC
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
//
15+
#include "koladata/contrib/flatten_cyclic_references.h"
16+
#include "koladata/data_slice_qtype.h"
17+
#include "koladata/internal/op_utils/qexpr.h"
18+
19+
namespace koladata::contrib {
20+
namespace {
21+
22+
KODA_QEXPR_OPERATOR("kd_ext.contrib._flatten_cyclic_references",
23+
FlattenCyclicReferences,
24+
"kd_ext.contrib.flatten_cyclic_references");
25+
26+
} // namespace
27+
} // namespace koladata::contrib

‎koladata/operators/BUILD‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,6 @@ cc_library(
8888
"curves.cc",
8989
"dicts.cc",
9090
"entities.cc",
91-
"flatten_cyclic_references.cc",
9291
"ids.cc",
9392
"lists.cc",
9493
"masking.cc",
@@ -116,7 +115,6 @@ cc_library(
116115
"curves.h",
117116
"dicts.h",
118117
"entities.h",
119-
"flatten_cyclic_references.h",
120118
"ids.h",
121119
"lists.h",
122120
"masking.h",
@@ -175,7 +173,6 @@ cc_library(
175173
"//koladata/internal/op_utils:qexpr",
176174
"//koladata/internal/op_utils:reverse",
177175
"//koladata/internal/op_utils:select",
178-
"//koladata/internal/op_utils:traverse_helper",
179176
"@com_google_absl//absl/algorithm:container",
180177
"@com_google_absl//absl/base:core_headers",
181178
"@com_google_absl//absl/base:no_destructor",

‎koladata/operators/operators.cc‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,6 @@
4141
#include "koladata/operators/curves.h"
4242
#include "koladata/operators/dicts.h"
4343
#include "koladata/operators/entities.h"
44-
#include "koladata/operators/flatten_cyclic_references.h"
4544
#include "koladata/operators/ids.h"
4645
#include "koladata/operators/json.h"
4746
#include "koladata/operators/json_stream.h"
@@ -101,8 +100,6 @@ OPERATOR_FAMILY("kd.core._attrs_impl",
101100
OPERATOR("kd.core._clone", Clone, "kd.core.clone");
102101
OPERATOR("kd.core._deep_clone", DeepClone, "kd.core.deep_clone");
103102
OPERATOR("kd.core._extract", Extract, "kd.core.extract");
104-
OPERATOR("kd.core._flatten_cyclic_references",
105-
FlattenCyclicReferences, "kd.core.flatten_cyclic_references");
106103
OPERATOR("kd.core._get_attr", GetAttr, "kd.core.get_attr");
107104
OPERATOR("kd.core._get_attr_with_default", GetAttrWithDefault,
108105
"kd.core.get_attr_with_default");

‎py/koladata/BUILD‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ py_library(
109109
"//py/koladata/ext:py_cloudpickle",
110110
"//py/koladata/ext:vis",
111111
"//py/koladata/ext/contrib:functions",
112-
"//py/koladata/ext/operators:kde_operators",
112+
"//py/koladata/ext/contrib:kde_operators",
113113
"//py/koladata/ext/pseudo_random",
114114
"//py/koladata/ext/storage",
115115
"//py/koladata/ext/view:kv",

‎py/koladata/dynamic_deps/BUILD‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,20 @@ cc_shared_library(
5151
],
5252
)
5353

54+
cc_shared_library(
55+
name = "koladata_operators_so",
56+
dynamic_deps = [
57+
"//py/koladata/dynamic_deps:koladata_so",
58+
],
59+
exports_filter = [
60+
"//py/koladata/operators:__subpackages__",
61+
],
62+
visibility = ["//visibility:public"],
63+
deps = [
64+
"//py/koladata/operators:cc_operators",
65+
],
66+
)
67+
5468
bzl_library(
5569
name = "py_extension_bzl",
5670
srcs = ["py_extension.bzl"],

0 commit comments

Comments
 (0)