Giter Club home page Giter Club logo

secretflow / spu Goto Github PK

View Code? Open in Web Editor NEW
201.0 8.0 88.0 12.82 MB

SPU (Secure Processing Unit) aims to be a provable, measurable secure computation device, which provides computation ability while keeping your private data protected.

Home Page: https://www.secretflow.org.cn/docs/spu/en/

License: Apache License 2.0

Shell 0.10% Python 18.63% C++ 78.47% MLIR 2.05% C 0.04% Markdown 0.48% Cuda 0.22%
privacy-preserving private-set-intersection secure-multiparty-computation

spu's Introduction

CircleCI

简体中文｜English

SecretFlow is a unified framework for privacy-preserving data intelligence and machine learning. To achieve this goal, it provides:

  • An abstract device layer consists of plain devices and secret devices which encapsulate various cryptographic protocols.
  • A device flow layer modeling higher algorithms as device object flow and DAG.
  • An algorithm layer to do data analysis and machine learning with horizontal or vertical partitioned data.
  • A workflow layer that seamlessly integrates data processing, model training, and hyperparameter tuning.

Documentation

SecretFlow Related Projects

  • Kuscia: A lightweight privacy-preserving computing task orchestration framework based on K3s.
  • SCQL: A system that allows multiple distrusting parties to run joint analysis without revealing their private data.
  • SPU: A provable, measurable secure computation device, which provides computation ability while keeping your private data protected.
  • HEU: A high-performance homomorphic encryption algorithm library.
  • YACL: A C++ library that contains cryptography, network and io modules which other SecretFlow code depends on.

Install

Please check INSTALLATION.md

Deployment

Please check DEPLOYMENT.md

Learn PETs

We also provide a curated list of papers and SecretFlow's tutorials on Privacy-Enhancing Technologies (PETs).

Please check AWESOME-PETS.md

Contributing

Please check CONTRIBUTING.md

Benchmarks

Please check OVERALL_BENCHMARK.md

Disclaimer

Non-release versions of SecretFlow are prohibited from using in any production environment due to possible bugs, glitches, lack of functionality, security issues or other problems.

spu's People

Contributors

6fj avatar 982945902 avatar anakinxc avatar deadlywing avatar duanyufi avatar elleryqu avatar fanguojie avatar fionser avatar gystar avatar hacker-jerry avatar jingshi-ant avatar krout0n avatar liang-xiaojian avatar lwxxxxxxx avatar magic-hya avatar mingbo-lee avatar nantsing avatar oahcnauygnid avatar rainysponge avatar renovate[bot] avatar salazar1117 avatar suelay avatar tarantula-leo avatar tongke6 avatar tpppppub avatar wangyixianggogo avatar winnylyc avatar ye-d avatar zhanglei486 avatar zoestartover avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

6fj tailm zhongmairen iamrealwilson justsoso0601 xfap pmpleader sukermao shifengbj skyer2 zhanglei486 playerxq zarca krout0n zhaochf xiaoqingzh rivertalk primummobile ychy00001 duanyufi anakinmac ant1proton tpppppub suelay chinchilla-forest f7ed imwangyt shazi4399 zouhuan1215 juljos anakinxc oeqqwq shaojian-ant lwxxxxxxx circlekkk cnt-ai oahcnauygnid mingbo-lee hacker-jerry tarantula-leo vincehong deadlywing magic-hya liangofthechen fanguojie llcurious zoestartover salazar1117 tututt nantsing lightbulb128 ye-d mspublic tonywu6 loki-chen rainysponge 982945902 web-logs2 elleryqu chiyee ues1df liang-xiaojian fionser gystar rinoex commis stu-yang takeminesoce wenyanliu wangyixianggogo nickzhong789 xiaohei-info winnylyc bestrongok v2hack tongke6 leonechen jamie-cui cryptoot lizzy-0323 maths644311798 qaz123ly notinghere

spu's Issues

libspu/compiler/main中调用了错误的重载函数

spu/libspu/compiler/main.cc

Line 55 in c5e761e

auto ret = spu::compiler::compile(context.get(), in_xla, "hlo");

这行代码中传入compile函数的第二个参数代表文件的路径，应该调用的函数是

std::string compile(CompilationContext *ctx,  
                              const std::filesystem::path &file);

但是会错误的调用函数

std::string compile(CompilationContext *ctx, const std::string &serialized_ir,
                    const std::string &ir_type);

Error in compiling "psi_demo" file

I want to use the underlying SDK of psi and pir. At present, the compilation environment is installed in the official image.
When compiling and executing "psi_demo", an error is reported and there is no clue.

The specific implementation process is as follows:

cd spu
bazel build //libspu/psi/demo:psi_demo

Error reporting segment：
image

在做PSI时，如果一方运行，另一方不运行。在尝试连接超时时会出现段错误

As shown in the figure below, when doing PSI, if one side is running, the other side is not running. A "Aborted (core dumped)" occurs when an attempt to connect times out. At the same time, after modifying this problem (that is, modifying the code to ignore the thrown exception), it is easy to cause a "Aborted (core dumped)" when one party continues to not receive data. Guess it was caused by throwing an exception.
image

ositioning problem:

  1. yasl/link/transport/channel.cc ： 86
  2. yasl/yasl/link/context.cc ：141

BeaverTfpUnsafe Reconstruct

I don't know much about cryptography. In spu/mpc/beaver/beaver_tfp.cc, I don't understand the operation for the third party to do:
if (lctx_->Rank() == 0) { c = tp_.adjustDot(descs, M, N, K); }
In my perspective, two parties first generate random arrays a, b, and c respectively. These random arrays are different for the two parties. I used to think the c of Rank 1 adding the adjusted c of Rank 0 is the original multiplication result of a and b of Rank 1. But I didn't get the expected results in validation.
What's more, I notice that the reconstruct operation needs the current seed and PrgArrayDesc arrays. What is the relation between the PRG arrays and the reconstructed r0, rs? If the secret I want to reconstruct is related to a complex combination of different arrays which are not stored in a PrgArrayDesc array. Do I need to do extra efforts to copy the elements in different arrays to a new PrgArrayDesc array?

Gcc 11: error: 'this' pointer is null [-Werror=nonnull]

When I use command: bazel test //...

I got this error:

ERROR: /root/work/secretflow/secretflow_lib/third_party/spu/spu/hal/BUILD.bazel:279:12: Compiling spu/hal/permute_util_test.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 124 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
spu/hal/permute_util_test.cc: In static member function 'static spu::hal::Value spu::hal::{anonymous}::PermuteTest_1d_Test::TestBody()::<lambda(spu::HalContext*, const spu::hal::Value&)>::_FUN(spu::HalContext*, const spu::hal::Value&)':
spu/hal/permute_util_test.cc:42:3: error: 'this' pointer is null [-Werror=nonnull]
42 | };
| ^
spu/hal/permute_util_test.cc:40:28: note: in a call to non-static member function 'spu::hal::{anonymous}::PermuteTest_1d_Test::TestBody()::<lambda(spu::HalContext*, const spu::hal::Value&)>'
40 | auto permute_wrapper_3 = [](HalContext* ctx, const Value& input) {
| ^
spu/hal/permute_util_test.cc: In static member function 'static spu::hal::Value spu::hal::{anonymous}::PermuteTest_1d_Test::TestBody()::<lambda(spu::HalContext*, const spu::hal::Value&)>::_FUN(spu::HalContext*, const spu::hal::Value&)':
spu/hal/permute_util_test.cc:39:3: error: 'this' pointer is null [-Werror=nonnull]
39 | };
| ^
spu/hal/permute_util_test.cc:37:28: note: in a call to non-static member function 'spu::hal::{anonymous}::PermuteTest_1d_Test::TestBody()::<lambda(spu::HalContext*, const spu::hal::Value&)>'
37 | auto permute_wrapper_2 = [](HalContext* ctx, const Value& input) {
| ^
spu/hal/permute_util_test.cc: In static member function 'static spu::hal::Value spu::hal::{anonymous}::PermuteTest_1d_Test::TestBody()::<lambda(spu::HalContext*, const spu::hal::Value&)>::_FUN(spu::HalContext*, const spu::hal::Value&)':
spu/hal/permute_util_test.cc:35:3: error: 'this' pointer is null [-Werror=nonnull]
35 | };
..........
..........

Security check in malicious security protocol

Hi, I'm now trying to implement SpdzWiseField protocol into spu and I am now analysing the feasibility and designing the code. I noticed that there isn't malicious security protocol in spu yet, so here is a problem that where to do the security check is good and efficient. Putting them into the end of kernels like MulAA is proper? Or setting a batch size to check them in batches? Thanks!

Have you considered implementing ABY2.0 protocol?

Hi, :)
I'm noticed that the ABY2.0 protocol has constant round in online communication, which is very effective for Secure Two-Party Computation.
Have you considered implementing this protocol? When i'm benchmarking Cheetah, i don't think it's fast enougth, maybe this protocol is more efficiently?
BTW, because SPU has JIT compilation, i'm curious about how to divide offline & online process.

Simulation error with jax median operation

Hi,
I simulate the jax median function with the following code and want to see pphlo dialect, but get error.
The code is:

import spu
import jax.numpy as jnp
import numpy as np
import spu.binding.util.simulation as pps

protocol = spu.ProtocolKind.ABY3
field = spu.FieldType.FM64
simulator = pps.Simulator.simple(3, protocol, field)

x = np.array([1,2])
y = np.array([3,4])

def add(x, y):
    tmp = jnp.concatenate((x, y), axis=0)
    return jnp.median(tmp)

spu_add = pps.sim_jax(simulator, add)
print(spu_add(x, y))
print(spu_add.pphlo)

The error is:

WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
Traceback (most recent call last):
  File "test_hlo.py", line 18, in <module>
    print(spu_sigmoid(x, y))
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 145, in wrapper
    out_flat = sim(executable, *args_flat)
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 100, in __call__
    parties = [job.join() for job in jobs]
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 100, in <listcomp>
    parties = [job.join() for job in jobs]
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 43, in join
    raise self.exc
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 36, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/simulation.py", line 89, in wrapper
    rt.run(executable)
  File "/root/miniconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/api.py", line 45, in run
    return self._vm.Run(executable.SerializeToString())
RuntimeError: what: 
        [Enforce fail at spu/hal/constants.cc:72] v.storage_type().isa<mpc::Pub2kTy>(). got aby3.BShr<PT_U8,1>
stacktrace: 
#0 spu::hal::test::dump_public_as<>()+0x7f8428fe92ad
#1 spu::device::getConditionValue()+0x7f8428feab98
#2 spu::device::pphlo::kernel::Sort<>()::{lambda()#1}::operator()()::{lambda()#2}::operator()()+0x7f8428fb9ec5
#3 std::__insertion_sort<>()+0x7f8428fba216
#4 std::__stable_sort_adaptive<>()+0x7f8428fbafca
#5 spu::device::pphlo::RegionExecutor::execute()+0x7f8428fbc423
#6 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fad100
#7 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fad526
#8 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fad7d6
#9 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fada86
#10 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fadd36
#11 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fadfe6
#12 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fae296
#13 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fae546
#14 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428fae796
#15 spu::device::pphlo::RegionExecutor::dispatchOp<>()+0x7f8428faea46

Question about the Type system

Hi, I am reading the code about the definition of secret share, and in spu/mpc/aby3/type.h: 22, AShrTy inherits from TypeImpl<AShrTy, RingTy, Secret, AShare>. I followed into TypeImpl and here are my questions:

  • Why the first item in the templates is the class itself? Is that a stipulation?
  • Is RingTy means this data is in ring?
  • Is Secret means this data is secret instead of public?
  • Is AShare emphasizes that this is an Arithmetic data? In other words, the point of it is Arithmetic instead of Share?
  • Is TypeImpl just defined the attribute of the share, and it doesn't store any real data? For example, there's a replicated secret share (x0, x1), and TypeImpl only defines attribute such as RingTy, Secret, etc. but don't store x0, x1.

Thanks for reply!

Error in Running Python example millionare.py and script build_and_packaging.sh

Hi, I am using the official docker image and installed spu from pypi with version 0.3.0. Now I am trying to run the examples/python/millionare.py that provided in the repo. But I got the error log:

Traceback (most recent call last):
  File "millionare.py", line 39, in <module>
    ppd.init(conf["nodes"], conf["devices"])
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 1099, in init
    _CONTEXT = HostContext(nodes_def, devices_def)
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 1026, in __init__
    self.devices[name] = SPU(
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 941, in __init__
    results = [future.result() for future in futures]
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 941, in <listcomp>
    results = [future.result() for future in futures]
  File "/usr/lib64/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib64/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 251, in run
    return self._call(self._stub.Run, fn, *args, **kwargs)
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 240, in _call
    rsp_data = rebuild_messages(rsp_itr.data for rsp_itr in rsp_gen)
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 220, in rebuild_messages
    return b''.join([msg for msg in msgs])
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 220, in <listcomp>
    return b''.join([msg for msg in msgs])
  File "/usr/local/lib64/python3.8/site-packages/spu/binding/util/distributed.py", line 240, in <genexpr>
    rsp_data = rebuild_messages(rsp_itr.data for rsp_itr in rsp_gen)
  File "/usr/local/lib64/python3.8/site-packages/grpc/_channel.py", line 426, in __next__
    return self._next()
  File "/usr/local/lib64/python3.8/site-packages/grpc/_channel.py", line 826, in _next
    raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "failed to connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused"
        debug_error_string = "UNKNOWN:Failed to pick subchannel {created_time:"2022-12-20T05:10:53.619911+00:00", children:[UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: Failed to connect to remote host: Connection refused {created_time:"2022-12-20T05:10:53.6199095+00:00", grpc_status:14}]}"

Another error occurs when I am trying to install spu from source, which need to run
sh build_and_packaging.sh -i
but I got:

[root@6cd766c57e0c spu]# sh build_and_packaging.sh -i
Start build wheel package...
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
WARN[0000] "/" is not a shared mount, this could cause issues or missing mounts with rootless containers 
cannot clone: Operation not permitted
Error: cannot re-exec process
Build exited with non-zero.

And the script terminated. It says "cannot clone: Operation not permitted", but I ran this as root.

Thanks, looking forward for reply.

Questions about unbalanced-psi benchmark

hi there, I've seen your post on https://zhuanlan.zhihu.com/p/616130287 about unbalanced psi benchmark and would like to try it out by myself.

I followed the instruction: created conda, activated conda, started ray(alice as head and bob as worker-node linked to it), and generated 20 million vs 2 billion ids (with 10 million intersection) csv files as alice and bob's data to make the intersection (since I did not found any of the test datasets published). After which I tried to run the offline script copied from the code, changed the IP, port, and paths to the local files (and I was wondering what is the difference between "dummy.alice.csv" and "unbalanced_2000w.csv" or "dummy.bob.csv" and "unbalanced_200000w.csv"?)

2023-03-27 17:25:17,397	INFO worker.py:1352 -- Connecting to existing Ray cluster at address: *.*.*.10:9394...
2023-03-27 17:25:17,405	INFO worker.py:1538 -- Connected to Ray cluster.
=====offline phase====
Traceback (most recent call last):
  File "offline.py", line 96, in <module>
    app.run(main)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "offline.py", line 74, in main
    reports = spu.psi_csv(
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 1231, in psi_csv
    return dispatch(
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/device/device/register.py", line 111, in dispatch
    return _registrar.dispatch(self.device_type, name, self, *args, **kwargs)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/device/device/register.py", line 80, in dispatch
    return self._ops[device_type][name](*args, **kwargs)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/device/kernels/spu.py", line 263, in psi_csv
    return sfd.get(res)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/distributed/primitive.py", line 49, in get
    return ray.get(object_refs)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/ray/_private/worker.py", line 2309, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::SPURuntime.psi_csv() (pid=2315511, ip=*.*.*.11, repr=<secretflow.device.device.spu.SPURuntime object at 0x7f0cda835bb0>)
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/secretflow/device/device/spu.py", line 547, in psi_csv
    psi_type=psi.PsiType.Value(protocol),
  File "/root/anaconda3/envs/sf-benchmark/lib/python3.8/site-packages/google/protobuf/internal/enum_type_wrapper.py", line 73, in Value
    raise ValueError('Enum {} has no value defined for name {!r}'.format(
ValueError: Enum PsiType has no value defined for name 'ECDH_OPRF_UB_PSI_2PC_OFFLINE'

Hope to get help for this. Thank you so much!

隐语的老师们好，之前在知乎上看到您公开的 非平衡psi benchmark，想动手自己试一试。因为没看到公开的测试用数据集，我尝试自己生成了所需的 alice 和 bob 分别两千万和二十个亿的 id、有一千万交集的测试用csv数据集，conda 里安装好secretflow，启动 ray，并尝试执行文章中 offline 部分的脚本，得到了如上的报错。

（事实上我还比较好奇，脚本中，dummy.alice.csv 和 unbalanced_2000w.csv，dummy.bob.csv 和 unbalanced_2000000w.csv 区别到底是什么？）

想请教您关于这个错误应该如何解决？谢谢！

请问PIR例子中的oprf_key.bin是如何生成的？

Issue Type

Documentation Bug

Modules Involved

PIR

Have you reproduced the bug with SPU HEAD?

Yes

Installation Kind

source

SPU Version

lastest

OS Platform and Distribution

No response

Python Version

No response

Compiler Version

No response

Current Behavior?

想咨询下https://www.secretflow.org.cn/docs/spu/en/development/pir.html
中提到的oprf_key.bin是如何生成的？文档并未说明

Standalone code to reproduce the issue

如题

Relevant log output

No response

Questions about the performance of serialization and deserialization using Protobuf

Hi,
I'm reading your wonderful code, and find that the binding of C++ secret values to Python object is implemented by Protobuf.
The serialization and deserialization may cost hundreds of milliseconds, but i'm benchmarking the performance of SSLR model and find that training speed is very fast.
So i'm curious about that is there any optimization in the serialization and deserialization?

源码安装出现错误

我想用源码安装secureflow，在执行python setup.py bdist_wheel时，出现以下错误：
image

我的机器环境 centos 7.9, 已安装bazel 5.11、devtoolset-11、cmake 4.2.1

目前spu有长时间监听一个rpc端口服务的example吗

从目前提供的例子（psi）来看，都是临时监听本地端口，然后去连接远程rpc端口。代码里好像都是这么写的。CreateContext的时候会去监听本地端口。
yacl::link::FactoryBrpc factory;
link_ctx = factory.CreateContext(lctx_desc, self_rank);
link_ctx->ConnectToMesh();

实际使用过程中是要考虑并行多任务的例子，所以应该需要一个全局保持监听的服务，目前有如何使用yacl的例子吗？

Weird jax operations in spu

Issue Type

Bug

Source

binary

Secretflow Version

latex

OS Platform and Distribution

Ubuntu 22.04

Python version

3.8.15

Bazel version

No response

GCC/Compiler version

No response

What happend and What you expected to happen.

When running jax code in spu, the jax jit requires array boolean indices must be concrete, so I tried two methods, both failed. And I can't understand the error of method2.

Reproduction code to reproduce the issue.

*****method1*****
for j in range(n_clusters):
    centroids = centroids.at[j].set(X[labels == j].mean(axis=0))

*****method2*****(The jax official documents says using where() func instead of boolean mask can avoid NonConcreteBooleanIndexError)
centroids = jnp.array([jnp.mean(X[jnp.where(labels == j)], axis=0) for j in range(n_clusters)])

*****error of method1*****
jax._src.traceback_util.UnfilteredStackTrace: jax._src.errors.NonConcreteBooleanIndexError: Array boolean indices must be concrete; got ShapedArray(bool[100])

*****error of method2*****(I think this error is very weird)
The size argument of jnp.nonzero must be statically specified to use jnp.nonzero within JAX transformations.

Some questions about the implementation of ABY3 and it's Truncation protocol

Hi,
I'm reading the implementation code of ABY3 and meet some problems.
The implementation of TruncA follows the protocol introduced by ABY3, that is:
image
But what I am confused about is that the right shift operation(as well as division) on the ring element isn't homomorphic, which means (x1'+x2'+x3')/2^d may not equal to (x1'+x2')/2^d+(x3'/2^d).
I'm curious about how the SPU Team solved this problem?
Thanks~

unbalanced psi报错

spu版本： 0.3.2b12
运行脚本 example/python/psi/unbalanced_psi.py
环境：centOS7.9

目前出现2个问题

  1. broadcast_result 设置为True时，receiver_rank一方有正常的交集，另一方显示交集为0
  2. 某些情况下出现如下bug：
    image

看代码对应是这一段主动抛出的异常，请问是什么原因触发？

 while (in->GetLine(&line)) {
    if (idx < header_line_count) {
      out->Write(line);
      out->Write("\n");
    } else {
      if (indices_iter == indices.end()) {
        break;
      }
      if (*indices_iter == idx - header_line_count) {
        indices_iter++;
        out->Write(line);
        out->Write("\n");
        ++target_count;
      }
    }
    idx++;
  }

  SPU_ENFORCE(target_count == indices.size(),
              "logstic error, indices.size={}, target_count={}, please be "
              "sure the `indices` is sorted",
              indices.size(), target_count);

能否给部分计算过程单独分配资源?

在16c32g配置下进行测试，发现SPU编译阶段耗时较长，ray: spu_compile 似乎是主要需求cpu的进程，但实际监视发现CPU总占用率较低，ray:: spu_compile这样的进程只占用了其中一个cpu，是否有方法可以指定给这个过程多分配一些资源？

top - 11:01:25 up 12:18,  1 user,  load average: 1.11, 1.08, 1.12
Tasks: 310 total,   2 running, 308 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.7 us,  0.5 sy,  6.4 ni, 92.3 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  32102.7 total,  13923.8 free,  12820.9 used,   5357.9 buff/cache
MiB Swap:  22048.0 total,  22048.0 free,      0.0 used.  18793.5 avail Mem

    PID        PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND  
  70431        35  15   26.3g  10.5g 190268 R  99.7  33.3  21:24.65 ray::_s+
   1046        20   0 6850572 323336 129076 S   7.0   1.0   0:58.00 gnome-s+
  70164        20   0   13.0g 381584 190452 S   3.3   1.2   0:44.32 python3  
  70421        35  15   11.2g  83776  39184 S   1.0   0.3   0:10.13 ray::ID+
  70424        35  15   11.2g  83956  39348 S   1.0   0.3   0:10.07 ray::ID+
  70430        35  15   11.2g  83844  39224 S   1.0   0.3   0:10.20 ray::ID+
  70434        35  15   11.2g  83568  38984 S   1.0   0.3   0:10.39 ray::ID+
  71132        35  15   13.2g 362688 187220 S   1.0   1.1   0:12.98 ray::SP+
  71141        35  15   13.2g 360472 184892 S   1.0   1.1   0:12.85 ray::SP+
   1786        20   0  566512  57080  43124 S   0.7   0.2   0:03.95 gnome-t+
  70198        20   0 1729620  30688  13304 S   0.7   0.1   0:08.44 gcs_ser+
  70299        20   0   20.5g  43136  14616 S   0.7   0.1   0:10.73 raylet   
  70337        20   0 1059004  70672  32376 S   0.7   0.2   0:10.24 python3  
  70420        35  15   11.2g  83744  39152 S   0.7   0.3   0:10.12 ray::ID+
  70423        35  15   11.2g  83012  38376 S   0.7   0.3   0:10.19 ray::ID+
  70425        35  15   16.2g 361040 189076 S   0.7   1.1   0:11.96 ray::ID+
  70426        35  15   11.2g  83724  39128 S   0.7   0.3   0:10.17 ray::ID+

资源消耗如上，在spu中定义了num_cpus=16

spu编译出错

在 docker 中运行bazel build //examples/cpp/pir -c opt
只有这两个文件报了 404，其他的都能下载

WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/0aaa6e679847a4eeb407136e7b0bcef93ec652e6.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/081771d4a0e9d7d3aa0eed2ef389fa4700dfb23e.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: Skipping '//examples/cpp/pir': no such target '//examples/cpp/pir:pir': target 'pir' not declared in package 'examples/cpp/pir' defined by /home/admin/dev/examples/cpp/pir/BUILD.bazel
WARNING: Target pattern parsing failed.
ERROR: no such target '//examples/cpp/pir:pir': target 'pir' not declared in package 'examples/cpp/pir' defined by /home/admin/dev/examples/cpp/pir/BUILD.bazel
INFO: Elapsed time: 218.512s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded)

然后就报warning了
请问如何处理？

Questions about the codes in makeAby3Protocol

Hi, I'm recently learning how to create a new mpc protocol in spu. I read the makeAby3Protocol function in spu/mpc/aby3/protocol.cc: 29 and have some questions:

  1. What is aby3::registerTypes() doing? Is it something about register the secret sharing type in arithmetic and binary?
  2. What is regABKernels() doing? I saw and guess from the codes inside that it registers operations between plain and secret, but what's the difference with spu/mpc/aby3/protocol.cc:51 and below? (obj->regKernel<aby3::P2A>(), obj->regKernel<aby3::AddAP>(), etc.) Is this the relation like api and kernel? regABKernels() registers the api and for example obj->regKernel<aby3::AddAP>() is the underlying implementation.

Thanks for reply!

Start nodes error

When I ran the start nodes command for running ss_lr.py, I got the following error:

File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/execroot/spulib/bazel-out/k8-opt/bin/examples/python/utils/nodectl.runfiles/spulib/spu/binding/util/distributed.py", line 312, in RunReturn
result = fn(self, *args, **kwargs)
File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/execroot/spulib/bazel-out/k8-opt/bin/examples/python/utils/nodectl.runfiles/spulib/spu/binding/util/distributed.py", line 262, in builtin_fetch_object
return server._globals[ObjectRef(refid, server.node_id)]
KeyError: ObjRef(c7e739ae-8a73-4556-ac11-49b57c8d85e4 at node:4)`

Besides, I didn't use GPU. Does it matter?

static_argnames in spu

Issue Type

Feature Request

Source

binary

Secretflow Version

latex

OS Platform and Distribution

Ubuntu 22.04

Python version

3.8.15

Bazel version

No response

GCC/Compiler version

No response

What happend and What you expected to happen.

What is the use of static_argnames in SPU?

Reproduction code to reproduce the issue.

...

代码智能提示

spu在0.3.0版本更新时移除了对hedron_compile_commands的依赖。请问在新版本的代码中应该通过何种方式正确地生成spu的compile_commands.json文件？

代码智能提示

spu在0.3.0版本更新时移除了对hedron_compile_commands的依赖。请问在新版本的代码中应该通过何种方式正确地生成spu的compile_commands.json文件？

centos7.6 使用bazel build 报错

环境满足readme的要求，但 bazel build //... -c opt 报错如下：

ERROR: /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/external/com_github_intel_ipp/BUILD.bazel:10:20: Foreign Cc - CMake: Building ipp failed: (Exit 1): bash failed: error executing command /bin/bash -c bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/wrapper_build_script.sh

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
rules_foreign_cc: Build failed!
rules_foreign_cc: Keeping temp build directory and dependencies directory for debug.
rules_foreign_cc: Please note that the directories inside a sandbox are still cleaned unless you specify --sandbox_debug Bazel command line flag.
rules_foreign_cc: Printing build logs:
_____ BEGIN BUILD LOGS _____

Bazel external C/C++ Rules. Building library ipp

Environment:______________
TMPDIR=/tmp
BUILD_WRAPPER_SCRIPT=bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/wrapper_build_script.sh
EXT_BUILD_ROOT=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib
INSTALLDIR=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp
PATH=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
BUILD_TMPDIR=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.build_tmpdir
PWD=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib
EXT_BUILD_DEPS=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps
SHLVL=3
BUILD_LOG=bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/CMake.log
BUILD_SCRIPT=bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/build_script.sh
_=/usr/bin/env
__________________________
+ cmake -DARCH=intel64 -DOPENSSL_INCLUDE_DIR=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps/openssl/include -DOPENSSL_LIBRARIES=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps/openssl/lib -DOPENSSL_ROOT_DIR=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps/openssl -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.build_tmpdir/crosstool_bazel.cmake -DCMAKE_INSTALL_PREFIX=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp -DCMAKE_PREFIX_PATH=/root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps -DCMAKE_RANLIB= -DCMAKE_MAKE_PROGRAM=ninja -G Ninja /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp
-- The C compiler identification is GNU 11.2.0
-- The CXX compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/gcc - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python: /usr/bin/python (found version "3.8.13") found components: Interpreter 
-- CMAKE_VERSION ......................... 3.24.1
-- NONPIC_LIB ............................ off
-- PROJECT ............................... Intel® Integrated Performance Primitives Cryptography (Intel® IPP Cryptography)
-- CMAKE_BINARY_DIR ...................... /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.build_tmpdir
-- CMAKE_OUTPUT_DIR ...................... /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.build_tmpdir/.build
-- CMAKE_SOURCE_DIR ...................... /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp
-- IPP_CRYPTO_DIR ........................ /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp
-- CMAKE_GENERATOR ....................... Ninja
-- CMAKE_C_COMPILER_ID ................... GNU
-- CMAKE_CXX_COMPILER_ID ................. GNU
-- IPP_CRYPTO_INCLUDE_DIR ................ /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp/include
-- IPP_CRYPTO_SOURCES_INCLUDE_DIR ........ /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp/sources/include
-- IPP_CRYPTO_SOURCES_DIR ................ /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp/sources/ippcp
-- ARCH .................................. intel64
-- DYNAMIC_LIB ........................... ON
-- CMAKE_INSTALL_PREFIX .................. /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp
-- PYTHON_VERSION_STRING ................. 3.8.13
-- MERGED_BLD ............................ on
-- BUILD_EXAMPLES ........................ off
-- Using compiler options from ........... /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/external/com_github_intel_ipp/sources/cmake/linux/GNU8.2.0.cmake
-- The ASM_NASM compiler identification is NASM
-- Found assembler: /usr/bin/nasm
-- ASM compiler version .................. /usr/bin/nasm
-- ASM object format ..................... elf64
-- CMAKE_BUILD_TYPE is not set to Debug explicitly, defaulting to Release
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Could NOT find Threads (missing: Threads_FOUND) 
-- Found OpenSSL: /root/.cache/bazel/_bazel_root/9de4a6aef89daf40df580afb13ff9f9d/sandbox/processwrapper-sandbox/1092/execroot/spulib/bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp.ext_build_deps/openssl/lib/libcrypto.a (found suitable version "1.1.1l", minimum required is "1.1.0")  
-- Configuring done
CMake Error at /usr/local/share/cmake-3.24/Modules/FindOpenSSL.cmake:123 (set_property):
  The link interface of target "OpenSSL::Crypto" contains:

    Threads::Threads

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

Call Stack (most recent call first):
  /usr/local/share/cmake-3.24/Modules/FindOpenSSL.cmake:641 (_OpenSSL_target_add_dependencies)
  sources/ippcp/crypto_mb/CMakeLists.txt:73 (find_package)


-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.
_____ END BUILD LOGS _____
rules_foreign_cc: Build wrapper script location: bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/wrapper_build_script.sh
rules_foreign_cc: Build script location: bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/build_script.sh
rules_foreign_cc: Build log location: bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/CMake.log

ERROR: /opt/spu/examples/cpp/BUILD.bazel:58:14 Linking examples/cpp/simple_in_memory_psi failed: (Exit 1): bash failed: error executing command /bin/bash -c bazel-out/k8-opt/bin/external/com_github_intel_ipp/ipp_foreign_cc/wrapper_build_script.sh

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
INFO: Elapsed time: 157.103s, Critical Path: 36.62s
INFO: 793 processes: 16 internal, 777 processwrapper-sandbox.
FAILED: Build did NOT complete successfully

求教，谢谢~

RuntimeError：indicies value must be public

Hi,
I ran my code on SPU-0.3.2b5 , the following error occurred:
RuntimeError: what: \n\t[Enforce fail at libspu/kernel/hlo/utils.cc:42] (value.isPublic()). indicies value must be public.
Part of my code:

        j=jnp.argmax(self.cache)
        temp=self.calc(j)
        self.cache=self.cache.at[j].set(temp)

There seems to be an error because of 'j'
Can I use 'j' as an array index in SPU?

spu源码编译出错

老师你好，我按照install.md的指导，进行的源码编译。

我看shell脚本,是直接下拉了一个有编译环境的docker镜像，但是我在编译过程出现了错误如下，请问应该怎么解决呢？

(base) zhanghao@thinkcentre:~/Downloads/mycode/spu$ ./build_and_packaging.sh
Start build wheel package...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-38
creating build/lib.linux-x86_64-cpython-38/spu
copying spu/api.py -> build/lib.linux-x86_64-cpython-38/spu
copying spu/psi.py -> build/lib.linux-x86_64-cpython-38/spu
copying spu/__init__.py -> build/lib.linux-x86_64-cpython-38/spu
copying spu/version.py -> build/lib.linux-x86_64-cpython-38/spu
creating build/lib.linux-x86_64-cpython-38/spu/utils
copying spu/utils/frontend.py -> build/lib.linux-x86_64-cpython-38/spu/utils
copying spu/utils/distributed.py -> build/lib.linux-x86_64-cpython-38/spu/utils
copying spu/utils/simulation.py -> build/lib.linux-x86_64-cpython-38/spu/utils
copying spu/utils/__init__.py -> build/lib.linux-x86_64-cpython-38/spu/utils
running build_ext
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: Rule 'yacl' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1676014912 +0800"
DEBUG: Repository yacl instantiated at:
  /home/admin/dev/WORKSPACE:19:9: in <toplevel>
  /home/admin/dev/bazel/repositories.bzl:45:10: in spu_deps
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule git_repository defined at:
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'llvm-raw' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/xla/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'tf_runtime' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/xla/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'llvm-raw' because it already exists.
Loading: 0 packages loaded
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/410c7f3b07f1d1170b13242a2cf9af4ec7edd33f.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'pybind11_bazel' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_github_grpc_grpc' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'zlib' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'rules_python' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'upb' because it already exists.
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: spu/utils ... (2 packages)
Analyzing: 3 targets (2 packages loaded, 0 targets configured)
INFO: Repository local_execution_config_python instantiated at:
  /home/admin/dev/WORKSPACE:55:15: in <toplevel>
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/xla/workspace2.bzl:43:19: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/workspace2.bzl:610:19: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/workspace2.bzl:61:27: in _tf_toolchains
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/tools/toolchains/remote_config/configs.bzl:6:28: in initialize_rbe_configs
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/tools/toolchains/remote_config/rbe_config.bzl:158:27: in _tensorflow_local_config
Repository rule local_python_configure defined at:
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl:279:41: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_execution_config_python':
   Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl", line 216, column 39, in _create_local_python_repository
		numpy_include = _get_numpy_include(repository_ctx, python_bin) + "/numpy"
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl", line 190, column 19, in _get_numpy_include
		return execute(
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/remote_config/common.bzl", line 230, column 13, in execute
		fail(
Error in fail: Problem getting numpy include path.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
Is numpy installed?
ERROR: /home/admin/dev/WORKSPACE:55:15: fetching local_python_configure rule //external:local_execution_config_python: Traceback (most recent call last):
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl", line 216, column 39, in _create_local_python_repository
		numpy_include = _get_numpy_include(repository_ctx, python_bin) + "/numpy"
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl", line 190, column 19, in _get_numpy_include
		return execute(
	File "/root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/remote_config/common.bzl", line 230, column 13, in execute
		fail(
Error in fail: Problem getting numpy include path.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
Is numpy installed?
INFO: Repository local_config_python instantiated at:
  /home/admin/dev/WORKSPACE:55:15: in <toplevel>
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/xla/workspace2.bzl:43:19: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/workspace2.bzl:610:19: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/workspace2.bzl:71:21: in _tf_toolchains
Repository rule python_configure defined at:
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/third_party/py/python_configure.bzl:298:35: in <toplevel>
INFO: Repository rules_cc instantiated at:
  /home/admin/dev/WORKSPACE:59:15: in <toplevel>
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/xla/workspace1.bzl:11:19: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/tsl/workspace1.bzl:17:28: in workspace
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/rules_cuda/cuda/dependencies.bzl:72:18: in rules_cuda_dependencies
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/rules_cuda/cuda/dependencies.bzl:35:17: in _rules_cc
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/eceb46742416a02f6a0f8d92bc74468c/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
ERROR: Analysis of target '//spu:init' failed; build aborted: Problem getting numpy include path.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
Is numpy installed?
INFO: Elapsed time: 81.430s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (52 packages loaded, 107 targets configured)
FAILED: Build did NOT complete successfully (52 packages loaded, 107 targets configured)
Traceback (most recent call last):
  File "setup.py", line 219, in <module>
    setuptools.setup(
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/root/miniconda3/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
    self.run_command('build')
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 132, in run
    self.run_command(cmd_name)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/root/miniconda3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "setup.py", line 193, in run
    return pip_run(self)
  File "setup.py", line 170, in pip_run
    build(True, True)
  File "setup.py", line 136, in build
    return bazel_invoke(
  File "setup.py", line 104, in bazel_invoke
    result = invoker(['bazel'] + cmdline, *args, **kwargs)
  File "/root/miniconda3/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bazel', 'build', '--verbose_failures', '-c', 'opt', '--', '//spu:init', '//spu/utils:distributed', '//spu:api']' returned non-zero exit status 1.
Build exited with non-zero.

请教一下一些基本的算子问题（加减）

假设目前有两个节点a,b，双方各自有一个数组，分别是x，y.我想加他们相加得到结果z，然后我通过spu::kernel::hal::test::dump_public_as 函数得到解密后的结果rz，但是经过我的测试，我在a端对b节点的数组y进行spu::kernel::hal::test::dump_public_as操作，也能得到正常的结果。这样是不是会造成a，b端都可以获取对方的数据？

（这里加法是举个例子，我的意思是一些复杂操作也能获取对方的数据）
下面是我的代码：

auto hctx = MakeHalContext(parties_,parma_.rank); spu::device::ColocatedIo cio(hctx.get()); std::vector<int> vec; for(int i=0;i<10;++i) { vec.push_back(parma_.rank*10+i); } cio.hostSetVar(fmt::format("x-{}", hctx->lctx()->Rank()), vec); cio.sync(); auto x = cio.deviceGetVar("x-0"); auto y = cio.deviceGetVar("x-1"); auto z = spu::kernel::hal::add(hctx.get(),x,y); xt::xarray<int> rx = spu::kernel::hal::test::dump_public_as<int>( hctx.get(), spu::kernel::hal::reveal(hctx.get(), x)); xt::xarray<int> ry = spu::kernel::hal::test::dump_public_as<int>( hctx.get(), spu::kernel::hal::reveal(hctx.get(), y)); xt::xarray<int> rz = spu::kernel::hal::test::dump_public_as<int>( hctx.get(), spu::kernel::hal::reveal(hctx.get(), z));

spu 有网络相关的文档吗？

主要是c++接口上的，从目前运行的几个psi example上看都是本地交互的，我想知道有没有两个远程机器交互的例子或者文档？（其实就是如何设置远端的i p和port的）。
另外，spu是如何使用brpc有相关的文档吗？

How did you perform the comparison protocol in SPU

Dear authors:
Thank you for the wonderful code. I was wondering how you implemented the comparison protocol in the SPU, i haven't found the corresponding code. The protocol for comparison is LTZ, did you used this prototol? Looking for your reply!

Owner of input data in P2A

Hi, In the P2A::proc of arithmetic.cc, how do I know the owner of the input for some index? Or it has assumed all the inputs are from party0? If this is the case, then I think it isn't also support for joint input?

cond控制流似乎不能正常使用

Issue Type

Bug

Source

binary

Secretflow Version

0.7.7b1

OS Platform and Distribution

18.04

Python version

3.8.13

Bazel version

No response

GCC/Compiler version

No response

What happend and What you expected to happen.

写了一个小demo测试一下cond能否在SPU上使用，cond_test()函数的作用类似于relu，大于0的保持不变，小于0的变为0。在jit上能运行，但到了SPU上就报错。可以帮忙看一下什么原因吗？

loc(fused["xla_computation(spu_test)/jit(main)/jit(f)/cond[linear=(False, False)]", "jit_test.py":25:0]): error: failed to legalize operation 'mhlo.case'
Traceback (most recent call last):
  File "jit_test.py", line 58, in <module>
    data = spu_test(data1, data2)
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/distributed.py", line 663, in __call__
    executable, args_flat, out_tree = self._compile_jax_func(
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/distributed.py", line 730, in _compile_jax_func
    executable, output = spu_fe.compile(
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/util/frontend.py", line 87, in compile
    mlir = spuapi.compile(xla, input_vis)
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/api.py", line 146, in compile
    return _spu_compilation(xla, MessageToJson(spu_pb2.XlaMeta(inputs=vis)))
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/cachetools/__init__.py", line 641, in wrapper
    v = func(*args, **kwargs)
  File "/home/ndbtjay/anaconda3/envs/secretflow/lib/python3.8/site-packages/spu/binding/api.py", line 131, in _spu_compilation
    return _lib.compile(xla, json_meta, "")
RuntimeError: what: 
        [spu/compiler/front_end/fe.cc:50] Run front end pipeline failed
stacktrace: 
#0 spu::compiler::compile()+0x7f9f5a4eeaa4
secretflow/secretflow#1 pybind11::cpp_function::initialize<>()::{lambda()#3}::_FUN()+0x7f9f5a4c9ee6
secretflow/secretflow#2 pybind11::cpp_function::dispatcher()+0x7f9f5a4d0552
secretflow/secretflow#3 cfunction_call_varargs+0x55ad948e300e

Reproduction code to reproduce the issue.

import argparse
import json
import jax
from jax import random, jit
import jax.numpy as jnp
import spu.binding.util.distributed as ppd

parser = argparse.ArgumentParser(description='distributed driver.')
parser.add_argument("-c", "--config", default="/home/ndbtjay/Desktop/PPC/SecretFlow/spu/examples/python/conf/3pc.json")
args = parser.parse_args()

with open(args.config, 'r') as file:
    conf = json.load(file)

ppd.init(conf["nodes"], conf["devices"])

def true_func(data, idx):
    return data

def false_func(data, idx):
    data = data.at[idx].set(0)
    return data

def f(data, idx):
    return jax.lax.cond(data[idx]>0, true_func, false_func, data, idx)

def cond_test(data):
    idx = 0
    for num in data:
        data = jax.jit(f)(data, idx)
        idx += 1
    return data

@ppd.device("SPU")
def spu_test(data1, data2):
    data = jnp.concatenate((data1, data2), axis=0) # 横向联邦
    data = cond_test(data)
    return data

def get_data(data):
    return data

if __name__=="__main__":
    # run on cpu
    data = [1, 2, 3, -3, -2, -1, 4, 5, 6]
    data = jnp.asarray(data)
    data = cond_test(data)
    print(data)
    
    # run on spu
    data = [1, 2, 3, -3, -2, -1, 4, 5, 6]
    data = jnp.asarray(data)
    n_points = len(data)
    data1 = data[: (n_points//2)]
    data2 = data[(n_points//2) :]
    data1 = ppd.device("P1")(get_data)(data1)
    data2 = ppd.device("P2")(get_data)(data2)
    data = spu_test(data1, data2)
    data = ppd.get(data)
    print(data)

启动ppd集群失败

在终端运行
python -m spu.utils.distributed up

出现以下错误

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/root/miniconda3/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/root/yp_workplace/ht_workplace/spu_try/spu/spu/__init__.py", line 18, in <module>
    from .spu_pb2 import (  # type: ignore
ModuleNotFoundError: No module named 'spu.spu_pb2'

相应位置具体代码为

from .spu_pb2 import (  # type: ignore
    DataType,
    Visibility,
    PtType,
    ProtocolKind,
    FieldType,
    ValueProto,
    ShapeProto,
    RuntimeConfig,
    ExecutableProto,
)

以上结果为使用pip install spu 安装时出现

但在尝试使用sh build_and_packaging.sh -i 安装时
出现以下错误

build_and_packaging.sh: 20: Syntax error: "(" unexpected

具体代码为

function build_spu() {
    echo -e "${RED}Start build wheel package...${NC}"

    if [[ "$OSTYPE" == "linux-gnu"* ]]; then
        docker run --rm --mount type=bind,source="$(pwd)",target=/home/admin/dev/ \
            -w /home/admin/dev \
            --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
            --cap-add=NET_ADMIN \
            --privileged=true \
            --entrypoint "./build_wheel_entrypoint.sh" \
            registry.hub.docker.com/secretflow/release-ci:latest
    elif [[ "$OSTYPE" == "darwin"* ]]; then
        sh ./build_wheel_entrypoint.sh
    fi

    (($? != 0)) && {
        echo -e "${RED}Build exited with non-zero.${NC}"
        exit 1
    }
}
...

simulation中pps.sim_jax的static_argnums参数如何使用？

当我运行这样一段代码时，有报错信息：

import jax
import jax.numpy as jnp
import spu
import spu.binding.util.simulation as pps

protocol = spu.ProtocolKind.ABY3
field = spu.FieldType.FM128
config = spu.RuntimeConfig(protocol=protocol, field=field)
config.enable_pphlo_profile = True
config.enable_action_trace = True
simulator = pps.Simulator(3, config)

def test_add(x, y, x0=20, y0=30):
    return x + x0, y + y0

if __name__ == "__main__":
    spu_test_add = pps.sim_jax(simulator,test_add,static_argnums=(2,3))
    x = 0
    y = 10
    x1, y1 = spu_test_add(x,y,20,30)

报错信息为：

Traceback (most recent call last):
  File "**/sim.py", line 21, in <module>
    x1, y1 = spu_test_add(x,y,20,30)
  File "**/python3.8/site-packages/spu/binding/util/simulation.py", line 128, in wrapper
    f, dyn_args = japi_util.argnums_partial_except(
  File "**/python3.8/site-packages/jax/_src/api_util.py", line 180, in argnums_partial_except
    return _argnums_partial(f, dyn_argnums, tuple(fixed_args)), dyn_args
  File "**/python3.8/site-packages/jax/linear_util.py", line 219, in transformation
    return fun.wrap(gen, gen_static_args, None)
AttributeError: 'function' object has no attribute 'wrap'

我尝试跟踪了下报错信息，但没清楚一个函数的wrap函数是怎么来的

jax.random.randint在spu环境下结果似乎不能正确运行

Issue Type

Bug

Source

binary

Secretflow Version

0.7.18b0

OS Platform and Distribution

Linux centos 3.10.0

Python version

3.8.13

Bazel version

No response

GCC/Compiler version

No response

What happend and What you expected to happen.

根据jax文档，jax.random.randint函数可以根据key生成一个可以选定范围大小的数字，但是在spu环境下，这个范围大小似乎不成立。
如下面给出的代码，尝试循环100个epoch，生成100个在[0,5)范围的随机数，统计超出范围的随机数个数。
在jit环境下，返回的统计个数为0；而在spu环境下，返回的统计个数却是100。这是为什么？

Reproduction code to reproduce the issue.

def run_on_cpu():
    def train():
        def loop_fun(_, carry):
            key, cnt = carry
            key, subkey = random.split(key)
            j = random.randint(subkey, shape=(), minval=0, maxval=5, dtype=jnp.int32)
            delta = jnp.where(jnp.any(jnp.array([j>=5, j<0])), 1, 0)
            cnt += delta
            return (key, cnt)
        key = random.PRNGKey(42)
        return lax.fori_loop(0, 100, loop_fun, (key, 0))
    key, cnt = jax.jit(train)()
    print(key, cnt)
# > [3622954601 1653733208] 0

# spu环境下
def run_on_spu():
    @ppd.device("SPU")
    def train():        
        def loop_fun(_, carry):
            key, cnt = carry
            key, subkey = random.split(key)
            j = random.randint(subkey, shape=(), minval=0, maxval=5, dtype=jnp.int32)
            delta = jnp.where(jnp.any(jnp.asarray([j>=5, j<0])), 1, 0)
            cnt += delta
            return (key, cnt)
        key = random.PRNGKey(42)
        key, cnt = lax.fori_loop(0, 100, loop_fun, (key, 0))
        return key, cnt
     key, cnt = train()
     key, cnt = ppd.get(key), ppd.get(cnt)
     print(key, cnt)
#> [3463333171 2461478805] 100

Why use Concept-Based Polymorphism in type system

Hi,
I read the implementation code of Type:

spu/spu/core/type.h

Line 173 in 1c73e7a

class Type final {

and find that this class can be treated as Adaptor of various type classes(e.g. VoidTy, PtTy, RingTy and so on), and they have same semantics, but no inheritance.
Why don't you directly use derived-based Polymorphism? In this way,Type is implemented as base class and have many virtual functions, various type classes can derive from the base class. A pointer to base class can call the function of derived class at the runtime.
I'm curious about the advantages of Concept-Based Polymorphism.
Thanks for your wonderful code.

Howto Help: examples/python/millionare.py under production scenario

I have 3 hosts: 192.168.1.180/181/182 with firewalld stopped.

I starts 3 ray cluster on the hosts:

ray start --head --node-ip-address="192.168.1.180" --port="46379" --include-dashboard=False --disable-usage-stats
ray start --head --node-ip-address="192.168.1.181" --port="46379" --include-dashboard=False --disable-usage-stats
ray start --head --node-ip-address="192.168.1.182" --port="46379" --include-dashboard=False --disable-usage-stats
cluster_config = {
    'parties': {
        'alice': {
            'address': '192.168.1.181:40000'
        },
        'bob': {
            'address': '192.168.1.182:40000'
        },
        'charlie': {
            'address': '192.168.1.180:40000'
        }
    }
}

bob.txt
charlie.txt
alice.txt

I rewrite the 'examples/python/millionare.py' and try to

  1. just run python code in charlie.txt - hang: not work
  2. run alice.txt and bob.txt python code first, then run charlie.txt - hang: not work
(venv) [secretflow@H0 pressure]$ python charlie.py 
2023-02-17 10:09:58,043 INFO worker.py:1352 -- Connecting to existing Ray cluster at address: 192.168.1.180:46379...
2023-02-17 10:09:58,058 INFO worker.py:1538 -- Connected to Ray cluster.
2023-02-17 10:09:59 INFO fed.barriers [charlie] --  RecverProxy was successfully created.
(RecverProxyActor pid=26351) INFO:fed.barriers:Successfully start Grpc service without credentials.
2023-02-17 10:10:00 INFO fed.barriers [charlie] --  SendProxy was successfully created.
z(<class 'secretflow.device.device.spu.SPUObject'> is a device object ref, we can not access it directly.
^CTraceback (most recent call last):
  File "charlie.py", line 72, in <module>
    print(f"z = {sf.reveal(z)}")
  File "/home/secretflow/venv/lib/python3.8/site-packages/secretflow/device/driver.py", line 151, in reveal
    all_object = sfd.get(all_object_refs)
  File "/home/secretflow/venv/lib/python3.8/site-packages/secretflow/distributed/primitive.py", line 47, in get
    return fed.get(object_refs)
  File "/home/secretflow/venv/lib/python3.8/site-packages/fed/api.py", line 361, in get
    values = ray.get(ray_refs)
  File "/home/secretflow/venv/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/home/secretflow/venv/lib/python3.8/site-packages/ray/_private/worker.py", line 2303, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/home/secretflow/venv/lib/python3.8/site-packages/ray/_private/worker.py", line 668, in get_objects
    data_metadata_pairs = self.core_worker.get_objects(
  File "python/ray/_raylet.pyx", line 1554, in ray._raylet.CoreWorker.get_objects
  File "python/ray/_raylet.pyx", line 195, in ray._raylet.check_status
KeyboardInterrupt

So, what's wrong with my code, config, operation or scenario?

How to reduce the time cost of networking round in SPU

Hi,
There are several operations which need at least log(k) networking rounds, and is hard to be parallelized, such as carry-out circuit. I think there exists time cost everytime you call the networking interface to broadcast the secret shares.
I'm wondering, when dealing with big dataset, how to reduce the time cost on this? Does bRPC can help to resolve the problem?

当bazel run时报错An error occurred during the fetch of repository 'com_github_eigenteam_eigen'

您好，我目前想在Ubuntu 20.04上运行secretflow/spu/examples/python/ml/ss_lr，当我安装README.md进行编译时报错：

root@CARC-20:~/yp_workplace/secretflow/spu/examples/python/ml/ss_lr# bazel run -c opt //examples/python/utils:nodectl -- up
Starting local Bazel server and connecting to it...
DEBUG: Rule 'yacl' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1676014912 +0800"
DEBUG: Repository yacl instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:19:9: in <toplevel>
  /root/yp_workplace/secretflow/spu/bazel/repositories.bzl:45:10: in spu_deps
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule git_repository defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'llvm-raw' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'tf_runtime' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'llvm-raw' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'pybind11_bazel' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'com_github_grpc_grpc' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'zlib' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'rules_python' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'pybind11' because it already exists.
DEBUG: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/third_party/repo.bzl:132:14: 
Warning: skipping import of repository 'upb' because it already exists.
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/lib/gtl/BUILD:100:11: in linkstatic attribute of cc_library rule @tsl//tsl/lib/gtl:map_util: setting 'linkstatic=1' is recommended if there are no object files
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:215:17: in cc_library rule @xla//xla/stream_executor:dnn_proto_cc_impl: Target '@xla//xla/stream_executor:dnn_proto_cc_impl' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:215:17: in cc_library rule @xla//xla/stream_executor:dnn_proto_cc_headers_only: Target '@xla//xla/stream_executor:dnn_proto_cc_headers_only' violates visibility of target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:1109:17: in cc_library rule @xla//xla:autotune_results_proto_cc_impl: Target '@xla//xla:autotune_results_proto_cc_impl' violates visibility of alias '@tsl//tsl/protobuf:autotuning_proto_cc' referring to target '@tsl//tsl/protobuf:autotuning_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:1109:17: in cc_library rule @xla//xla:autotune_results_proto_cc_headers_only: Target '@xla//xla:autotune_results_proto_cc_headers_only' violates visibility of target '@tsl//tsl/protobuf:autotuning_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/gpu/BUILD:53:17: in cc_library rule @xla//xla/service/gpu:backend_configs_cc_impl: Target '@xla//xla/service/gpu:backend_configs_cc_impl' violates visibility of alias '@xla//xla/stream_executor:dnn_proto_cc' referring to target '@xla//xla/stream_executor:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/gpu/BUILD:53:17: in cc_library rule @xla//xla/service/gpu:backend_configs_cc_headers_only: Target '@xla//xla/service/gpu:backend_configs_cc_headers_only' violates visibility of target '@xla//xla/stream_executor:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/platform/BUILD:1022:11: in cc_library rule @tsl//tsl/platform:float8: Target '@tsl//tsl/platform:float8' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:201:11: in cc_library rule @xla//xla:types: Target '@xla//xla:types' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:177:11: in cc_library rule @xla//xla/stream_executor:data_type: Target '@xla//xla/stream_executor:data_type' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/platform/BUILD:1012:11: in cc_library rule @tsl//tsl/platform:bfloat16: Target '@tsl//tsl/platform:bfloat16' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/cpu/BUILD:1047:11: in cc_library rule @xla//xla/service/cpu:runtime_single_threaded_matmul_impl: Target '@xla//xla/service/cpu:runtime_single_threaded_matmul_impl' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/platform/BUILD:967:11: in cc_library rule @tsl//tsl/platform:threadpool_interface: Target '@tsl//tsl/platform:threadpool_interface' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:6604:11: in cc_library rule @xla//xla/service:global_device_id: Target '@xla//xla/service:global_device_id' violates visibility of target '@tsl//tsl/lib/gtl:int_type'. Continuing because --nocheck_visibility is active
DEBUG: Rule 'org_interconnection' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1669271462 +0800"
DEBUG: Repository org_interconnection instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:28:10: in <toplevel>
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/yacl/bazel/repositories.bzl:44:10: in yacl_deps
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule git_repository defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:200:11: in cc_library rule @xla//xla/stream_executor:device_memory: Target '@xla//xla/stream_executor:device_memory' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:252:11: in cc_library rule @xla//xla/stream_executor:rng: Target '@xla//xla/stream_executor:rng' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:89:11: in cc_library rule @xla//xla/stream_executor:launch_dim: Target '@xla//xla/stream_executor:launch_dim' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:139:11: in cc_library rule @xla//xla/stream_executor:kernel_spec: Target '@xla//xla/stream_executor:kernel_spec' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:576:11: in cc_library rule @xla//xla/stream_executor:host_or_device_scalar: Target '@xla//xla/stream_executor:host_or_device_scalar' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:164:11: in cc_library rule @xla//xla/stream_executor:allocator_stats: Target '@xla//xla/stream_executor:allocator_stats' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:223:11: in cc_library rule @xla//xla/stream_executor:fft: Target '@xla//xla/stream_executor:fft' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:124:11: in cc_library rule @xla//xla/stream_executor:device_description: Target '@xla//xla/stream_executor:device_description' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:124:11: in cc_library rule @xla//xla/stream_executor:device_description: Target '@xla//xla/stream_executor:device_description' violates visibility of target '@tsl//tsl/lib/math:math_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:206:11: in cc_library rule @xla//xla/stream_executor:device_options: Target '@xla//xla/stream_executor:device_options' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/platform/BUILD:258:11: in cc_library rule @tsl//tsl/platform:status: Target '@tsl//tsl/platform:status' violates visibility of target '@tsl//tsl/platform/default:status'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/platform/default/BUILD:109:11: in cc_library rule @tsl//tsl/platform/default:env: Target '@tsl//tsl/platform/default:env' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:29:11: in cc_library rule @xla//xla/stream_executor:stream_executor_headers: Target '@xla//xla/stream_executor:stream_executor_headers' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:279:11: in cc_library rule @xla//xla:util: Target '@xla//xla:util' violates visibility of target '@tsl//tsl/lib/gtl:iterator_range'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:279:11: in cc_library rule @xla//xla:util: Target '@xla//xla:util' violates visibility of target '@tsl//tsl/lib/math:math_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:231:11: in cc_library rule @xla//xla/stream_executor:platform: Target '@xla//xla/stream_executor:platform' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:667:11: in cc_library rule @xla//xla/stream_executor:dnn: Target '@xla//xla/stream_executor:dnn' violates visibility of target '@tsl//tsl/lib/strings:proto_serialization'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:561:11: in cc_library rule @xla//xla/stream_executor:blas: Target '@xla//xla/stream_executor:blas' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:561:11: in cc_library rule @xla//xla/stream_executor:blas: Target '@xla//xla/stream_executor:blas' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/client/BUILD:46:11: in cc_library rule @xla//xla/client:padding: Target '@xla//xla/client:padding' violates visibility of target '@tsl//tsl/lib/math:math_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/host/BUILD:19:11: in cc_library rule @xla//xla/stream_executor/host:host_platform_id: Target '@xla//xla/stream_executor/host:host_platform_id' violates visibility of target '@xla//xla/stream_executor:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:712:11: in cc_library rule @xla//xla/stream_executor:temporary_device_memory: Target '@xla//xla/stream_executor:temporary_device_memory' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/rocm/BUILD:143:11: in cc_library rule @xla//xla/stream_executor/rocm:rocm_platform_id: Target '@xla//xla/stream_executor/rocm:rocm_platform_id' violates visibility of target '@xla//xla/stream_executor:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:344:11: in cc_library rule @xla//xla/stream_executor:multi_platform_manager: Target '@xla//xla/stream_executor:multi_platform_manager' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:667:11: in cc_library rule @xla//xla/stream_executor:dnn: Target '@xla//xla/stream_executor:dnn' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:285:11: in cc_library rule @xla//xla/stream_executor:executor_cache: Target '@xla//xla/stream_executor:executor_cache' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:667:11: in cc_library rule @xla//xla/stream_executor:dnn: Target '@xla//xla/stream_executor:dnn' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/BUILD:866:11: in cc_library rule @xla//xla:shape_tree: Target '@xla//xla:shape_tree' violates visibility of target '@tsl//tsl/lib/gtl:iterator_range'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:586:11: in cc_library rule @xla//xla/stream_executor:event: Target '@xla//xla/stream_executor:event' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/cuda/BUILD:41:11: in cc_library rule @xla//xla/stream_executor/cuda:cuda_platform_id: Target '@xla//xla/stream_executor/cuda:cuda_platform_id' violates visibility of target '@xla//xla/stream_executor:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:466:11: in cc_library rule @xla//xla/stream_executor:kernel: Target '@xla//xla/stream_executor:kernel' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:466:11: in cc_library rule @xla//xla/stream_executor:kernel: Target '@xla//xla/stream_executor:kernel' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:285:11: in cc_library rule @xla//xla/stream_executor:executor_cache: Target '@xla//xla/stream_executor:executor_cache' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:586:11: in cc_library rule @xla//xla/stream_executor:event: Target '@xla//xla/stream_executor:event' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:363:11: in cc_library rule @xla//xla/stream_executor:stream_executor_internal: Target '@xla//xla/stream_executor:stream_executor_internal' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:387:11: in cc_library rule @xla//xla/stream_executor:stream_executor_pimpl_header: Target '@xla//xla/stream_executor:stream_executor_pimpl_header' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:387:11: in cc_library rule @xla//xla/stream_executor:stream_executor_pimpl_header: Target '@xla//xla/stream_executor:stream_executor_pimpl_header' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:693:11: in cc_library rule @xla//xla/stream_executor:temporary_memory_manager: Target '@xla//xla/stream_executor:temporary_memory_manager' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:527:11: in cc_library rule @xla//xla/stream_executor:timer: Target '@xla//xla/stream_executor:timer' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:527:11: in cc_library rule @xla//xla/stream_executor:timer: Target '@xla//xla/stream_executor:timer' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:420:16: in cc_library rule @xla//xla/stream_executor:stream_executor_pimpl: Target '@xla//xla/stream_executor:stream_executor_pimpl' violates visibility of target '@xla//xla/stream_executor/platform:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:420:16: in cc_library rule @xla//xla/stream_executor:stream_executor_pimpl: Target '@xla//xla/stream_executor:stream_executor_pimpl' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:420:16: in cc_library rule @xla//xla/stream_executor:stream_executor_pimpl: Target '@xla//xla/stream_executor:stream_executor_pimpl' violates visibility of alias '@tsl//tsl/protobuf:dnn_proto_cc' referring to target '@tsl//tsl/protobuf:dnn_proto_cc_headers_only'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/stream_executor/BUILD:772:11: in cc_library rule @xla//xla/stream_executor:stream_executor_impl: Target '@xla//xla/stream_executor:stream_executor_impl' violates visibility of target '@tsl//tsl/protobuf:dnn_proto_cc_impl'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3823:11: in cc_library rule @xla//xla/service:computation_placer: Target '@xla//xla/service:computation_placer' violates visibility of target '@xla//xla/stream_executor:stream_executor'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3823:11: in cc_library rule @xla//xla/service:computation_placer: Target '@xla//xla/service:computation_placer' violates visibility of target '@xla//xla/stream_executor:platform'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3823:11: in cc_library rule @xla//xla/service:computation_placer: Target '@xla//xla/service:computation_placer' violates visibility of target '@xla//xla/stream_executor/cuda:cuda_platform_id'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3823:11: in cc_library rule @xla//xla/service:computation_placer: Target '@xla//xla/service:computation_placer' violates visibility of target '@xla//xla/stream_executor/host:host_platform_id'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3823:11: in cc_library rule @xla//xla/service:computation_placer: Target '@xla//xla/service:computation_placer' violates visibility of target '@xla//xla/stream_executor/rocm:rocm_platform_id'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/hlo/ir/BUILD:19:11: in cc_library rule @xla//xla/hlo/ir:hlo: Target '@xla//xla/hlo/ir:hlo' violates visibility of target '@tsl//tsl/lib/gtl:iterator_range'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/hlo/ir/BUILD:19:11: in cc_library rule @xla//xla/hlo/ir:hlo: Target '@xla//xla/hlo/ir:hlo' violates visibility of target '@tsl//tsl/lib/gtl:map_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:4084:11: in cc_library rule @xla//xla/service:logical_buffer: Target '@xla//xla/service:logical_buffer' violates visibility of target '@tsl//tsl/lib/gtl:int_type'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3387:11: in cc_library rule @xla//xla/service:dot_merger: Target '@xla//xla/service:dot_merger' violates visibility of target '@xla//xla/service/graphcycles:graphcycles'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:4072:11: in cc_library rule @xla//xla/service:buffer_value_containers: Target '@xla//xla/service:buffer_value_containers' violates visibility of target '@tsl//tsl/lib/gtl:compactptrset'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:5390:11: in cc_library rule @xla//xla/service:hlo_graph_dumper: Target '@xla//xla/service:hlo_graph_dumper' violates visibility of target '@tsl//tsl/lib/gtl:map_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:5390:11: in cc_library rule @xla//xla/service:hlo_graph_dumper: Target '@xla//xla/service:hlo_graph_dumper' violates visibility of target '@tsl//tsl/lib/io:zlib_compression_options'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:5390:11: in cc_library rule @xla//xla/service:hlo_graph_dumper: Target '@xla//xla/service:hlo_graph_dumper' violates visibility of target '@tsl//tsl/lib/io:zlib_outputbuffer'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:3920:11: in cc_library rule @xla//xla/service:hlo_cost_analysis: Target '@xla//xla/service:hlo_cost_analysis' violates visibility of target '@tsl//tsl/lib/gtl:map_util'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:4416:11: in cc_library rule @xla//xla/service:tuple_points_to_analysis: Target '@xla//xla/service:tuple_points_to_analysis' violates visibility of target '@tsl//tsl/lib/gtl:compactptrset'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:393:11: in cc_library rule @xla//xla/service:dump: Target '@xla//xla/service:dump' violates visibility of target '@tsl//tsl/lib/io:zlib_compression_options'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:393:11: in cc_library rule @xla//xla/service:dump: Target '@xla//xla/service:dump' violates visibility of target '@tsl//tsl/lib/io:zlib_outputbuffer'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/BUILD:393:11: in cc_library rule @xla//xla/service:dump: Target '@xla//xla/service:dump' violates visibility of target '@tsl//tsl/lib/strings:proto_serialization'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/framework/BUILD:297:11: in cc_library rule @tsl//tsl/framework:fixedpoint_types: Target '@tsl//tsl/framework:fixedpoint_types' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/framework/fixedpoint/BUILD:14:11: in cc_library rule @tsl//tsl/framework/fixedpoint:fixedpoint: Target '@tsl//tsl/framework/fixedpoint:fixedpoint' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/tsl/tsl/framework/contraction/BUILD:65:11: in cc_library rule @tsl//tsl/framework/contraction:eigen_contraction_kernel_with_mkl: Target '@tsl//tsl/framework/contraction:eigen_contraction_kernel_with_mkl' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
WARNING: /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/xla/service/cpu/BUILD:1061:11: in cc_library rule @xla//xla/service/cpu:runtime_single_threaded_matmul: Target '@xla//xla/service/cpu:runtime_single_threaded_matmul' violates visibility of alias '@eigen_archive//:eigen3' referring to target '@xla//third_party/eigen3:eigen3'. Continuing because --nocheck_visibility is active
INFO: Repository com_github_eigenteam_eigen instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:19:9: in <toplevel>
  /root/yp_workplace/secretflow/spu/bazel/repositories.bzl:39:32: in spu_deps
  /root/yp_workplace/secretflow/spu/bazel/repositories.bzl:289:10: in _com_github_eigenteam_eigen
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
WARNING: Download from https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Checksum was 0c8c490764f9c2a793133491adca0cd073b73e0bde965c68cbe58d91b5ed4261 but wanted 8586084f71f9bde545ee7fa6d00288b264a2b7ac3607b974e54d13e7162c1c72
ERROR: An error occurred during the fetch of repository 'com_github_eigenteam_eigen':
   Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl", line 125, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz] to /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/com_github_eigenteam_eigen/temp11944514553605750315/eigen-3.4.0.tar.gz: Checksum was 0c8c490764f9c2a793133491adca0cd073b73e0bde965c68cbe58d91b5ed4261 but wanted 8586084f71f9bde545ee7fa6d00288b264a2b7ac3607b974e54d13e7162c1c72
ERROR: /root/yp_workplace/secretflow/spu/WORKSPACE:19:9: fetching http_archive rule //external:com_github_eigenteam_eigen: Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl", line 125, column 45, in _http_archive_impl
                download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz] to /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/com_github_eigenteam_eigen/temp11944514553605750315/eigen-3.4.0.tar.gz: Checksum was 0c8c490764f9c2a793133491adca0cd073b73e0bde965c68cbe58d91b5ed4261 but wanted 8586084f71f9bde545ee7fa6d00288b264a2b7ac3607b974e54d13e7162c1c72
INFO: Repository stablehlo instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:55:15: in <toplevel>
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/workspace2.bzl:50:28: in workspace
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/workspace2.bzl:18:14: in _initialize_third_party
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/third_party/stablehlo/workspace.bzl:11:20: in repo
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/third_party/repo.bzl:136:21: in tf_http_archive
Repository rule _tf_http_archive defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/xla/third_party/repo.bzl:89:35: in <toplevel>
INFO: Repository com_github_facebook_zstd instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:19:9: in <toplevel>
  /root/yp_workplace/secretflow/spu/bazel/repositories.bzl:36:30: in spu_deps
  /root/yp_workplace/secretflow/spu/bazel/repositories.bzl:87:10: in _com_github_facebook_zstd
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
INFO: Repository com_github_emptoolkit_emp_tool instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:28:10: in <toplevel>
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/yacl/bazel/repositories.bzl:34:36: in yacl_deps
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/yacl/bazel/repositories.bzl:296:10: in _com_github_emptoolkit_emp_tool
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
INFO: Repository com_github_brpc_brpc instantiated at:
  /root/yp_workplace/secretflow/spu/WORKSPACE:28:10: in <toplevel>
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/yacl/bazel/repositories.bzl:20:26: in yacl_deps
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/yacl/bazel/repositories.bzl:52:10: in _com_github_brpc_brpc
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/bazel_tools/tools/build_defs/repo/http.bzl:355:31: in <toplevel>
ERROR: /root/yp_workplace/secretflow/spu/libspu/mpc/utils/BUILD.bazel:92:15: //libspu/mpc/utils:linalg depends on @com_github_eigenteam_eigen//:eigen3 in repository @com_github_eigenteam_eigen which failed to fetch. no such package '@com_github_eigenteam_eigen//': java.io.IOException: Error downloading [https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz] to /root/.cache/bazel/_bazel_root/334f4c68bae767008a9ac672daa07cfa/external/com_github_eigenteam_eigen/temp11944514553605750315/eigen-3.4.0.tar.gz: Checksum was 0c8c490764f9c2a793133491adca0cd073b73e0bde965c68cbe58d91b5ed4261 but wanted 8586084f71f9bde545ee7fa6d00288b264a2b7ac3607b974e54d13e7162c1c72
ERROR: Analysis of target '//examples/python/utils:nodectl' failed; build aborted: 
INFO: Elapsed time: 45.835s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (267 packages loaded, 15513 targets configured)
FAILED: Build did NOT complete successfully (267 packages loaded, 15513 targets configured)
    Fetching @simplest_ot; Cloning f40a33a37e3ff8cd81655c35237c177e358dc5b1 of https://github.com/secretflow/simplest-ot.git 23s
    Fetching https://github.com/facebook/zstd/releases/download/v1.5.0/zstd-1.5.0.tar.gz; 106,496B 19s

似乎问题出现在eigen3的安装上面，但当我通过sudo apt-get install libeigen3-dev安装后，运行bazel run -c opt //examples/python/utils:nodectl -- up依然会报上述错误，想问一下如何解决正确安装eigen3？

about PSI

做PSI的时候，对先对本地数据做去重处理吗？

Sort answer mismatch

Found by @imwangyt

Repro

x1d = np.arange(10, 0, -1)
y1d = np.arange(6, 16)
y1d[0] = 99
print(f'x1d = {x1d}')
print(f'y1d = {y1d}')

input = jnp.concatenate([x1d, y1d])

def fn(ar):
    iota = jax.lax.broadcasted_iota(np.int64, np.shape(ar), dimension=0)
    return jax.lax.sort_key_val(ar, iota)

z = fn(input)
print(f'cpu out = {z}')

spu_fn = ppsim.sim_jax(sim, fn)
z = spu_fn(input)

print(f'spu out = {z}')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.