facebookresearch / private-id Goto Github PK

A collection of algorithms that can do join between two parties while preserving the privacy of keys on which the join happens

License: Apache License 2.0

Rust 99.62% Dockerfile 0.18% Shell 0.19%

private-id's Introduction

Private-ID

Private-ID is a collection of algorithms to match records between two or more parties, while preserving the privacy of these records. We present multiple algorithms to do this---one of which does an outer join between parties, and others do inner or left join and then generate additive shares that can then be input to a Multi Party Compute system like CrypTen. Please refer to our paper for more details. The MultiKey Private-ID paper and the Delegated Private-ID paper extend Private-ID.

Build

Private-ID is implemented in Rust to take advantage of the language's security features and to leverage the encryption libraries that we depend on. It should compile with the nightly Rust toolchain.

The following should build and run the unit tests for the building blocks used by the protocols

cargo build --release
cargo test --release

Each protocol involves two (or more) parties and they have to be run in their own shell environment. We call one party Company and another party Partner. Some protocols also involve additional parties such as the Helper and the Shuffler.

Run the script at etc/example/generate_cert.sh to generate dummy_certs directory if you want to test protocol with TLS on local.

Build & Run With Docker Compose

The following, run each party in a different container:

Private-ID: docker compose --profile private-id up
Delegated Private Matching for Compute (DPMC): docker compose --profile dpmc up
Delegated Private Matching for Compute with Secure Shuffling (DSPMC): docker compose --profile dspmc up

By default, this will create datasets of 10 items each. To run with bigger datasets set the ENV_VARIABLE_FOR_SIZE environment variable. For example: ENV_VARIABLE_FOR_SIZE=100 docker compose --profile dpmc up will run DPMC with datasets of 100 items each.

Note, to run on an ARM machine modify the Dockerfile and add --platform=linux/amd64 to the two FROM lines (e.g., FROM --platform=linux/amd64 rust:latest AS build, FROM --platform=linux/amd64 debian:stable-slim AS privateid).

Private-ID

This protocol maps the email addresses from both parties to a single ID spine, so that same e-mail addresses map to the same key.

To run Company:

env RUST_LOG=info cargo run --release --bin private-id-server -- \
  --host 0.0.0.0:10009 \
  --input etc/example/email_company.csv \
  --stdout \
  --no-tls

To run Partner:

env RUST_LOG=info cargo run --release --bin private-id-client -- \
  --company localhost:10009 \
  --input etc/example/email_partner.csv \
  --stdout \
  --no-tls

Private-ID MultiKey

We extend the Private-ID protocol to match multiple identifiers. Please refer to our paper for more details.

To run Company:

env RUST_LOG=info cargo run --release --bin private-id-multi-key-server -- \
  --host 0.0.0.0:10009 \
  --input etc/example/private_id_multi_key/Ex1_company.csv \
  --stdout \
  --no-tls

To run Partner:

env RUST_LOG=info cargo run --release --bin private-id-multi-key-client -- \
  --company localhost:10009 \
  --input etc/example/private_id_multi_key/Ex1_partner.csv \
  --stdout \
  --no-tls

PS3I

This protocol does an inner join based on email addresses as keys and then generates additive share of a feature associated with that email address. The shares are generated in the designated output files as 64-bit numbers

To run Company:

env RUST_LOG=info cargo run --release --bin cross-psi-server -- \
  --host 0.0.0.0:10010 \
  --input etc/example/input_company.csv \
  --output etc/example/output_company.csv \
  --no-tls

To run Partner:

env RUST_LOG=info cargo run --release --bin cross-psi-client -- \
  --company localhost:10010 \
  --input etc/example/input_partner.csv \
  --output etc/example/output_partner.csv \
  --no-tls

PS3I XOR

This protocol does an inner join based on email addresses as keys and then generates XOR share of a feature associated with that email address. The shares are generated in the designated output files as 64-bit numbers

To run Company:

env RUST_LOG=info cargo run --release --bin cross-psi-xor-server -- \
  --host 0.0.0.0:10010 \
  --input etc/example/cross_psi_xor/input_company.csv \
  --output etc/example/cross_psi_xor/output_company \
  --no-tls

To run Partner:

env RUST_LOG=info cargo run --release --bin cross-psi-xor-client -- \
  --company localhost:10010 \
  --input etc/example/cross_psi_xor/input_partner.csv \
  --output etc/example/cross_psi_xor/output_partner \
  --no-tls

The --output option provides prefix for the output files that contain the shares. In this case, Company generates two files; output_company_company_feature.csv and output_company_partner_feature.csv. They contain Company's share of company and partner features respectively. Similarly, Partner generates two files; output_partner_company_feature.csv and output_partner_partner_feature.csv. They contain Partner's share of company and partner features respectively.

Thus output_company_company_feature.csv and output_partner_company_feature.csv are XOR shares of Company's features. Similarly, output_partner_company_feature.csv and output_partner_partner_feature.csv are XOR shares of Partner's features.

Private Join and Compute

This is an implementation of Google's Private Join and Compute protocol, that does a inner join based on email addresses and computes a sum of the corresponding feature for the Partner.

To run Company:

env RUST_LOG=info cargo run --release --bin pjc-server -- \
  --host 0.0.0.0:10011 \
  --input etc/example/pjc_company.csv \
  --stdout \
  --no-tls

To run Partner:

env RUST_LOG=info cargo run --release --bin pjc-client -- \
  --company localhost:10011 \
  --input etc/example/pjc_partner.csv \
  --stdout \
  --no-tls

SUMID

This is an implementation of 2-party version of Secure Universal ID protocol. This can work on multiple keys. In the current implementation, the merger party also assumes the role of one data party and the sharer party assumes the role of all the other data parties. The data parties are the .csv files show below

To run merger:

env RUST_LOG=info cargo run --release --bin suid-create-server -- \
  --host 0.0.0.0:10010 \
  --input etc/example/suid/Example1/DataParty2_input.csv \
  --stdout \
  --no-tls

To run client:

env RUST_LOG=info cargo run --release --bin suid-create-client -- \
  --merger localhost:10010 \
  --input etc/example/suid/Example1/DataParty1_input.csv \
  --input etc/example/suid/Example1/DataParty3_input.csv \
  --stdout \
  --no-tls

The output will be ElGamal encrypted Universal IDs assigned to each entry in the .csv file.

Delegated Private Matching for Compute (DPMC)

We extend the Multi-key Private-ID protocol to multiple partners. Please refer to our paper for more details.

To run Company:

env RUST_LOG=info cargo run --release --bin dpmc-company-server -- \
  --host 0.0.0.0:10010 \
  --input etc/example/dpmc/Ex0_company.csv \
  --stdout \
  --output-shares-path etc/example/dpmc/output_company \
  --no-tls

To run multiple partners (servers):

env RUST_LOG=info cargo run --release --bin dpmc-partner-server -- \
  --host 0.0.0.0:10020 \
  --company localhost:10010 \
  --input-keys etc/example/dpmc/Ex0_partner_1.csv \
  --input-features etc/example/dpmc/Ex0_partner_1_features.csv \
  --no-tls

env RUST_LOG=info cargo run --release --bin dpmc-partner-server -- \
  --host 0.0.0.0:10021 \
  --company localhost:10010 \
  --input-keys etc/example/dpmc/Ex0_partner_2.csv \
  --input-features etc/example/dpmc/Ex0_partner_2_features.csv \
  --no-tls

Start helper (client):

env RUST_LOG=info cargo run --release --bin dpmc-helper -- \
  --company localhost:10010 \
  --partners localhost:10020,localhost:10021 \
  --stdout \
  --output-shares-path etc/example/dpmc/output_partner \
  --no-tls

The above will generate one-to-one matches.

To explain the results, we need to look at the inputs first:

Inputs

Company Input:

email1
email2
email3
email4

Partner 1 Input (IDs):

email1
email7

Partner 1 Input (Associated Data):

10, 0
50, 50

Partner 2 Input:

email1
email4

Partner 2 Input (Associated Data):

20, 21
30, 31

Outputs

Company:

2C124C57A040C6FEB396F101F84C3B8C6A466FA53C0FDED94E8F725F2E9704B,email4
6695895CB82E629598547D93FA67403D4249B83A9944A21E53BBE3F9854F7140,email3
CEE0A32A239B802558ABFD57EE87587B5FD15D64E73FD805D13A1303CDD5429,email2
FEC9F87838BEEFFD3B689D13A538FB05767B2F9CDEE53903D22E67B91F,email1

Output Secret shares at etc/example/dpmc/output_company_partner_features.csv:

2123763108355018584,7917888405770470969
7553524091763063603,12192982022453250030
12025288841580037526,5628706741631442660
12193188557740602958,3696238821401023600

Helper:

2C124C57A040C6FEB396F101F84C3B8C6A466FA53C0FDED94E8F725F2E9704B, Partner enc key at pos 0
6695895CB82E629598547D93FA67403D4249B83A9944A21E53BBE3F9854F7140,NA
CEE0A32A239B802558ABFD57EE87587B5FD15D64E73FD805D13A1303CDD5429,NA
FEC9F87838BEEFFD3B689D13A538FB05767B2F9CDEE53903D22E67B91F, Partner enc key at pos 0

Output Secret shares at etc/example/dpmc/output_partner_partner_features.csv:

2123763108355018566,7917888405770470950
7553524091763063603,12192982022453250030
12025288841580037526,5628706741631442660
12193188557740602948,3696238821401023600

Since DPMC focuses on left-join, wherever was a match in Company's dataset, we have secret shares of the partner's associated data, while wherever there was no match, we have secret shares of zero.

Indeed, since email1 and email4 matched:

2123763108355018584 ^ 2123763108355018566 = 30,   7917888405770470969 ^ 7917888405770470950 = 31
7553524091763063603 ^ 7553524091763063603 = 0,    12192982022453250030 ^ 12192982022453250030 = 0
12025288841580037526 ^ 12025288841580037526 = 0,  5628706741631442660 ^ 5628706741631442660 = 0
12193188557740602958 ^ 12193188557740602948 = 10, 3696238821401023600 ^ 3696238821401023600 = 0

Observe that email1 matched with both partners but since this is one-to-one matching then the first match was only considered.

One-to-many matches

To enable one-to-many matches (one record from C will match to M P records), use the flag --one-to-many M in the dpmc-helper binary, where M is the number of matches.

For example, using the same scripts as above for company and partners, to run 1-2 matching, start the helper as follows:

env RUST_LOG=info cargo run --release --bin dpmc-helper -- \
  --company localhost:10010 \
  --partners localhost:10020,localhost:10021 \
  --one-to-many 2 \
  --stdout \
  --output-shares-path etc/example/dpmc/output_partner \
  --no-tls

Outputs

Company:

267549DEDFC9898B9ADB99278E86162155119ADBDCC1589F44E12EC66AD723,email2
40C1E76B6F2CF94B1B86D31FD9FB5C62B9114C85FC2AAAB59A6A1379044323,email1
44469BA5EBF28547491442BA88A996C91D2E5C1874BD56131FDE6FC2C19F95B,email4
725FAAA4E9862E5983979C85E58AA59347FF2C5C1AE0CC89201B34711588E957,email3

Output Secret shares at etc/example/dpmc/output_company_partner_features.csv:

15639158529780438101,10355320774873656494
13789343269605551875,7497287768912087672
1103603035954233860,16491667106643692030
16818785984424715268,17987764095998628258
5216582505071635321,17033543400689351118
9296137075950449950,6917021766104166842
1775928733629157667,2173601871347247126
10727446575062113091,6625868366339267723

Helper:

267549DEDFC9898B9ADB99278E86162155119ADBDCC1589F44E12EC66AD723, NA
267549DEDFC9898B9ADB99278E86162155119ADBDCC1589F44E12EC66AD723, NA
40C1E76B6F2CF94B1B86D31FD9FB5C62B9114C85FC2AAAB59A6A1379044323, Partner enc key at pos 0
40C1E76B6F2CF94B1B86D31FD9FB5C62B9114C85FC2AAAB59A6A1379044323, Partner enc key at pos 1
44469BA5EBF28547491442BA88A996C91D2E5C1874BD56131FDE6FC2C19F95B, Partner enc key at pos 0
44469BA5EBF28547491442BA88A996C91D2E5C1874BD56131FDE6FC2C19F95B, NA
725FAAA4E9862E5983979C85E58AA59347FF2C5C1AE0CC89201B34711588E957, NA
725FAAA4E9862E5983979C85E58AA59347FF2C5C1AE0CC89201B34711588E957, NA

Output Secret shares at etc/example/dpmc/output_partner_partner_features.csv:

15639158529780438101,10355320774873656494
13789343269605551875,7497287768912087672
1103603035954233870,16491667106643692030
16818785984424715280,17987764095998628279
5216582505071635303,17033543400689351121
9296137075950449940,6917021766104166842
1775928733629157667,2173601871347247126
10727446575062113091,6625868366339267723

Since DPMC focuses on left-join, wherever was a match in Company's dataset, we have secret shares of the partner's associated data, while wherever there was no match, we have secret shares of zero.

Indeed, since email1 and email4 matched:

15639158529780438101 ^ 15639158529780438101 = 0,  10355320774873656494 ^ 10355320774873656494 = 0
13789343269605551875 ^ 13789343269605551875 = 0,  7497287768912087672 ^ 7497287768912087672 = 0
1103603035954233860 ^ 1103603035954233870 = 10,   16491667106643692030 ^ 16491667106643692030 = 0
16818785984424715268 ^ 16818785984424715280 = 20, 17987764095998628258 ^ 17987764095998628279 = 21
5216582505071635321 ^ 5216582505071635303 = 30,   17033543400689351118 ^ 17033543400689351121 = 31
9296137075950449950 ^ 9296137075950449940 = 0,   6917021766104166842 ^ 6917021766104166842 = 0
1775928733629157667 ^ 1775928733629157667 = 0,    2173601871347247126 ^ 2173601871347247126 = 0
10727446575062113091 ^ 10727446575062113091 = 0,  6625868366339267723 ^ 6625868366339267723 = 0

Observe that email1 matched with both partners and here we have secret shares for both.

Delegated Private Matching for Compute with Secure Shuffling (DsPMC)

Start helper (server):

env RUST_LOG=info cargo run --release --bin dspmc-helper-server -- \
  --host 0.0.0.0:10030 \
  --stdout \
  --output-shares-path etc/example/dspmc/output_helper \
  --no-tls

Start company (server):

env RUST_LOG=info cargo run --release --bin dspmc-company-server -- \
  --host 0.0.0.0:10010 \
  --helper localhost:10030 \
  --input etc/example/dspmc/Ex0_company.csv \
  --stdout \
  --output-shares-path etc/example/dspmc/output_company \
  --no-tls

Start multiple partners (servers):

env RUST_LOG=info cargo run --release --bin dspmc-partner-server -- \
  --host 0.0.0.0:10020 \
  --company localhost:10010 \
  --input-keys etc/example/dspmc/Ex0_partner_1.csv \
  --input-features etc/example/dspmc/Ex0_partner_1_features.csv \
  --no-tls

env RUST_LOG=info cargo run --release --bin dspmc-partner-server -- \
  --host 0.0.0.0:10021 \
  --company localhost:10010 \
  --input-keys etc/example/dspmc/Ex0_partner_2.csv \
  --input-features etc/example/dspmc/Ex0_partner_2_features.csv \
  --no-tls

Start Shuffler (client):

env RUST_LOG=info cargo run --release --bin dspmc-shuffler -- \
  --company localhost:10010 \
  --helper localhost:10030 \
  --partners localhost:10020,localhost:10021 \
  --stdout \
  --no-tls

Note: Running over the network

To run over the network instead of localhost prepend the IP address with http:// or https://. For example:

To run Company (in IP 1.23.34.45):

env RUST_LOG=info cargo run --release --bin dpmc-company-server -- \
  --host 0.0.0.0:10010 \
  --input etc/example/dpmc/Ex0_company.csv \
  --stdout \
  --output-shares-path etc/example/dpmc/output_company \
  --no-tls

To run multiple partners (servers) (in IPs 76.65.54.43 and 76.65.54.44):

env RUST_LOG=info cargo run --release --bin dpmc-partner-server -- \
  --host 0.0.0.0:10020 \
  --company http://1.23.34.45:10010 \
  --input-keys etc/example/dpmc/Ex0_partner_1.csv \
  --input-features etc/example/dpmc/Ex0_partner_1_features.csv \
  --no-tls

env RUST_LOG=info cargo run --release --bin dpmc-partner-server -- \
  --host 0.0.0.0:10021 \
  --company http://1.23.34.45:10010 \
  --input-keys etc/example/dpmc/Ex0_partner_2.csv \
  --input-features etc/example/dpmc/Ex0_partner_2_features.csv \
  --no-tls

Start helper (client):

env RUST_LOG=info cargo run --release --bin dpmc-helper -- \
  --company http://1.23.34.45:10010 \
  --partners http://76.65.54.43:10020,http://76.65.54.44:10021 \
  --stdout \
  --output-shares-path etc/example/dpmc/output_partner \
  --no-tls

Citing Private-ID

To cite Private-ID in academic papers, please use the following BibTeX entries.

Delegated Private-ID

@Article{PoPETS:MMTSBC23,
  author = "Dimitris Mouris and
    Daniel Masny and
    Ni Trieu and
    Shubho Sengupta and
    Prasad Buddhavarapu and
    Benjamin M Case",
  title =   "{Delegated Private Matching for Compute}",
  volume =  2024,
  month =   Jul,
  year =    2024,
  journal = "{Proceedings on Privacy Enhancing Technologies}",
  number =  2,
  pages =   "1--24",
}

Multi-Key Private-ID

@Misc{EPRINT:BCGKMSTX21,
  author = "Prasad Buddhavarapu and
    Benjamin M Case and
    Logan Gore and
    Andrew Knox and
    Payman Mohassel and
    Shubho Sengupta and
    Erik Taubeneck and
    Min Xue",
  title = "Multi-key Private Matching for Compute",
  year = 2021,
  howpublished = "Cryptology ePrint Archive, Report 2021/770",
  note = "\url{https://eprint.iacr.org/2021/770}",
}

Private-ID

@Misc{EPRINT:BKMSTV20,
  author = "Prasad Buddhavarapu and
    Andrew Knox and
    Payman Mohassel and
    Shubho Sengupta and
    Erik Taubeneck and
    Vlad Vlaskin",
  title = "Private Matching for Compute",
  year = 2020,
  howpublished = "Cryptology ePrint Archive, Report 2020/599",
  note = "\url{https://eprint.iacr.org/2020/599}",
}

License

Private-ID is Apache 2.0 licensed, as found in the LICENSE file.

Additional Resources on Private Computation at Meta

private-id's People

Contributors

Stargazers

Watchers

private-id's Issues

Consider using double_and_compress_batch in encrypt_to_bytes.

I looked briefly at the code and noticed that the encrypt_to_bytes methods do a scalar multiplication and then a point compression for each input item. Although most of this work is the scalar multiplication, it's possible to amortize work across multiple point compressions, using the double_and_compress_batch method. The only wrinkle is that rather than encoding Q_i, this encodes 2Q_i. However, because Q_i is computed as k* P_i, it's still possible to use this method to get the same result as before, by multiplying k by (1/2) mod l (a constant value) to get k', and computing Q_i' <- k' * P_i. Since Q_i = 2 Q_i', batch-double-and-encoding the Q_i' will give enc(Q_1) , ... , enc(Q_n). This costs only one scalar-scalar multiplication for each batch, and I believe will save time as long as the batch is more than 1 element large.

The batching is not parallelizable, but this can still be applied in the parallel case by breaking a larger batch into chunks, and applying the optimization within each chunk.

Also, I think that the code could be slightly streamlined if the methods took impl Iterator<Item=...> rather than slices. For instance, if encrypt_to_bytes took an iterator, instead of having to have separate hash_encrypt_to_bytes, a caller could do

// plaintexts is an iterator of byte slices
encrypt_to_bytes(plaintexts.map(RistrettoPoint::hash_from_bytes::<Sha512>))

and the hash-to-curve calculations would be inlined into the right place.

Understanding PJC implementation

In the Private Join and Compute protocol, I don't understand the requirement for the company database to have an associated value to each keys and what their meaning is (0/1). The original implementation of PJC does not use such a value.

Build fails

Building this fails with this error:

Any ideas about what's causing this?

error[E0277]: the trait bound `BigInt: From<&[u8]>` is not satisfied
  --> crypto/src/he.rs:48:9
   |
48 |         BigInt::from(&*buf) >> (nun_bytes * BYTE - bit_size)
   |         ^^^^^^^^^^^^ the trait `From<&[u8]>` is not implemented for `BigInt`
   |
   = help: the following implementations were found:
             <BigInt as From<RawCiphertext<'b>>>
             <BigInt as From<RawPlaintext<'b>>>
             <BigInt as From<i32>>
             <BigInt as From<u32>>
             <BigInt as From<u64>>
   = note: required by `std::convert::From::from`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
  --> crypto/src/he.rs:52:12
   |
52 |         (a.mod_floor(modulus) + b.mod_floor(modulus)).mod_floor(modulus)
   |            ^^^^^^^^^ method not found in `&BigInt`
   |
   = help: items from traits can only be used if the trait is in scope
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
           `use num_integer::Integer;`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
  --> crypto/src/he.rs:52:35
   |
52 |         (a.mod_floor(modulus) + b.mod_floor(modulus)).mod_floor(modulus)
   |                                   ^^^^^^^^^ method not found in `&BigInt`
   |
   = help: items from traits can only be used if the trait is in scope
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
           `use num_integer::Integer;`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
  --> crypto/src/he.rs:56:24
   |
56 |         let sub_op = a.mod_floor(modulus) - b.mod_floor(modulus) + modulus;
   |                        ^^^^^^^^^ method not found in `&BigInt`
   |
   = help: items from traits can only be used if the trait is in scope
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
           `use num_integer::Integer;`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
  --> crypto/src/he.rs:56:47
   |
56 |         let sub_op = a.mod_floor(modulus) - b.mod_floor(modulus) + modulus;
   |                                               ^^^^^^^^^ method not found in `&BigInt`
   |
   = help: items from traits can only be used if the trait is in scope
   = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
           `use num_integer::Integer;`

error[E0599]: no function or associated item named `one` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:146:31
    |
146 |         let max_val = BigInt::one() << (PAILLIER_PUBLIC_KEY_SIZE / 2);
    |                               ^^^ function or associated item not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_traits::identities::One;`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
   --> crypto/src/he.rs:151:38
    |
151 |                 raw_ptext.0.as_ref().mod_floor(&max_val)
    |                                      ^^^^^^^^^ method not found in `&BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_integer::Integer;`

error[E0599]: no function or associated item named `one` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:157:31
    |
157 |         let max_val = BigInt::one() << (PAILLIER_PUBLIC_KEY_SIZE / 2);
    |                               ^^^ function or associated item not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_traits::identities::One;`

error[E0599]: no method named `mod_floor` found for reference `&BigInt` in the current scope
   --> crypto/src/he.rs:163:51
    |
163 |                 let val = RawPlaintext::from(kv.1.mod_floor(&max_val));
    |                                                   ^^^^^^^^^ method not found in `&BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_integer::Integer;`

error[E0599]: no function or associated item named `one` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:203:39
    |
203 |                 let max_val = BigInt::one() << (PAILLIER_PUBLIC_KEY_SIZE / 2);
    |                                       ^^^ function or associated item not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_traits::identities::One;`

error[E0599]: no method named `mod_floor` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:204:60
    |
204 |                 let neg_rhs = RawPlaintext::from(rhs.neg().mod_floor(&max_val));
    |                                                            ^^^^^^^^^ method not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_integer::Integer;`

error[E0599]: no method named `mod_floor` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:227:22
    |
227 |                     .mod_floor(&(BigInt::one() << (PAILLIER_PUBLIC_KEY_SIZE / 2)))
    |                      ^^^^^^^^^ method not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_integer::Integer;`

error[E0599]: no function or associated item named `one` found for struct `BigInt` in the current scope
   --> crypto/src/he.rs:227:42
    |
227 |                     .mod_floor(&(BigInt::one() << (PAILLIER_PUBLIC_KEY_SIZE / 2)))
    |                                          ^^^ function or associated item not found in `BigInt`
    |
    = help: items from traits can only be used if the trait is in scope
    = note: the following trait is implemented but not in scope; perhaps add a `use` for it:
            `use num_traits::identities::One;`

error: aborting due to 13 previous errors

Some errors have detailed explanations: E0277, E0599.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `crypto`

To learn more, run the command again with --verbose.

Build failure

➜ cargo --version     
cargo 1.48.0 (65cbdd2dc 2020-10-14)
➜ cargo +nightly --version
cargo 1.50.0-nightly (d274fcf86 2020-12-07)

➜ cargo +nightly build
...
error[E0432]: unresolved import `time`
  --> common/src/timer.rs:12:42
   |
12 | #[cfg(not(target_arch = "wasm32"))]  use time::Instant;
   |                                          ^^^^ help: a similar path exists: `std::time`

   Compiling curv v0.2.3 (https://github.com/KZen-networks/curv?tag=v0.2.3#d6c575b5)
   Compiling paillier v0.3.4 (https://github.com/KZen-networks/rust-paillier?tag=v0.3.4#564352cb)
warning: unused import: `ops::SubAssign`
 --> common/src/timer.rs:7:5
  |
7 |     ops::SubAssign,
  |     ^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

error: aborting due to previous error; 1 warning emitted

For more information about this error, try `rustc --explain E0432`.
error: could not compile `common`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

Issues while benchmarking performance

I am trying to benchmark the performance of Private-ID by conducting experiments. However, the running times I am getting are much higher as compared to those reported in the paper: "Private Matching for Compute". Do you have any ideas on why my times are significantly higher than those reported in the paper for similar experiments. Following are the details of the experiments I conducted:

Machine: Google Cloud Instance (e2-standard-32) (32 vCPUs and 128GB RAM)
Datasets size: 10,000 (Identifiers: 128 bit strings, values: 64 bit integers)
Intersection Size: 50%
The client and server processes run on a single machine.

Experiment 1:
Algorithm: Private-ID
Total time: 70.03021 sec (Single core), 5.16027 sec (Multi-core)

Experiment 2:
Algorithm: PS3I
Total time: 1259.84367 sec (multi-core)

These numbers are significantly higher than those reported by the paper for similar settings. i.e. For Private-ID, Table 1 shows that for 10,000 dataset size and single core, it takes 4.2s in contrast to 70.03 s that I get. Similarly, for PS3I, I get 1259.84s for 32 cores whereas the paper reports 247s for a single core. Do you have any thoughts why my benchmarking numbers are significantly higher?

Additionally, here is the log trace printed by the PS3I run:

[2022-07-12T18:57:36Z WARN rpc::connect::create_client] Connecting to company without TLS, avoid in production
[2022-07-12T18:57:36Z INFO rpc::connect::create_client] Connecting to host: http://localhost:10010/
[2022-07-12T18:57:36Z INFO rpc::connect::create_client] Client connected!
[2022-07-12T18:57:39Z INFO protocol::fileio] Data initialised with dimensions: cols: 1, rows: 10000, keys len: 10000
[2022-07-12T18:57:39Z INFO cross_psi_client] Number of company features 1
[2022-07-12T18:57:39Z INFO common::timer] [u_company_keys | received u_company_keys size: 10000] elapsed: 0.46777 sec [qps: 21378]
[2022-07-12T18:57:39Z INFO common::timer] [u_company_keys] elapsed: 0.93586 sec
[2022-07-12T18:57:39Z INFO common::timer] [e_company | keys EC enc + srlz size: 10000] elapsed: 0.00002 sec [qps: 571493885]
[2022-07-12T19:06:10Z INFO common::timer] [u_company_feature | received u_company_feature size: 10000] elapsed: 510.25326 sec [qps: 20]
[2022-07-12T19:06:10Z INFO common::timer] [u_company_feature] elapsed: 1020.50688 sec
[2022-07-12T19:08:37Z INFO cross_psi_client] e_company_ack feature index 0
[2022-07-12T19:08:37Z INFO common::timer] [u_partner | keys EC enc size: 10000] elapsed: 0.00002 sec [qps: 502537816]
[2022-07-12T19:17:20Z INFO common::timer] [u_partner | column 0 HE enc size: 10000] elapsed: 518.63875 sec [qps: 19]
[2022-07-12T19:17:20Z INFO cross_psi_client] e_company_ack feature_index 0
[2022-07-12T19:17:20Z INFO common::timer] [shares_company_indices | received shares_company_indices size: 5000] elapsed: 0.03449 sec [qps: 144985]
[2022-07-12T19:17:20Z INFO common::timer] [shares_company_indices] elapsed: 0.06919 sec
[2022-07-12T19:17:20Z INFO common::timer] [indices | recv company intersection indices size: 5000] elapsed: 0.03621 sec [qps: 138093]
[2022-07-12T19:17:20Z INFO cross_psi_client] company intersection size: 5000
[2022-07-12T19:17:21Z INFO common::timer] [shares_feature | received shares_feature size: 5000] elapsed: 0.02990 sec [qps: 167238]
[2022-07-12T19:17:21Z INFO common::timer] [shares_feature] elapsed: 0.05998 sec
[2022-07-12T19:17:21Z INFO protocol::cross_psi::partner] Saving self-shares for feature 0
[2022-07-12T19:18:36Z INFO common::timer] [indices | recv shares for feature 0 size: 5000] elapsed: 75.41446 sec [qps: 66]
[2022-07-12T19:18:36Z INFO protocol::cross_psi::partner] revealing columns to output file
[2022-07-12T19:18:36Z INFO common::timer] [global | total time size: 10000] elapsed: 1259.84367 sec [qps: 8]

Fails to compile because `zeroize` cannot be found

(ins)gorel:temp$ git clone https://github.com/facebookresearch/private-id
Cloning into 'private-id'...
remote: Enumerating objects: 184, done.
remote: Counting objects: 100% (184/184), done.
remote: Compressing objects: 100% (121/121), done.
remote: Total 184 (delta 68), reused 168 (delta 58), pack-reused 0
Receiving objects: 100% (184/184), 88.02 KiB | 1.47 MiB/s, done.
Resolving deltas: 100% (68/68), done.
(ins)gorel:temp$ cd private-id/
(ins)gorel:private-id$ cargo build --release
    Updating crates.io index
    Updating git repository `https://github.com/dalek-cryptography/curve25519-dalek.git`
    Updating git repository `https://github.com/KZen-networks/rust-paillier`
    Updating git repository `https://github.com/KZen-networks/curv`
    Updating git repository `https://github.com/KZen-networks/rust-gmp`
error: failed to select a version for the requirement `zeroize = "^0.10"`
candidate versions found which didn't match: 1.3.0, 1.2.0, 1.1.1, ...
location searched: crates.io index
required by package `curv v0.2.3 (https://github.com/KZen-networks/curv?tag=v0.2.3#d6c575b5)`
    ... which is depended on by `paillier v0.3.4 (https://github.com/KZen-networks/rust-paillier?tag=v0.3.4#564352cb)`
    ... which is depended on by `crypto v0.1.0 (/home/gorel/temp/private-id/crypto)`
    ... which is depended on by `protocol v0.1.0 (/home/gorel/temp/private-id/protocol)`
    ... which is depended on by `protocol-rpc v0.1.0 (/home/gorel/temp/private-id/protocol-rpc)`

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

facebookresearch / private-id Goto Github PK

private-id's Introduction

Private-ID

Build

Build & Run With Docker Compose

Private-ID

Private-ID MultiKey

PS3I

PS3I XOR

Private Join and Compute

SUMID

Delegated Private Matching for Compute (DPMC)

Inputs

Outputs

One-to-many matches

Outputs

Delegated Private Matching for Compute with Secure Shuffling (DsPMC)

Note: Running over the network

Citing Private-ID

Delegated Private-ID

Multi-Key Private-ID

Private-ID

License

Additional Resources on Private Computation at Meta

private-id's People

Contributors

Stargazers

Watchers

Forkers

private-id's Issues

Recommend Projects

Recommend Topics

Recommend Org