Comments (7)
SilverPoker
commented on July 30, 2024
The detailed error information of one of the failed tests is shown below:
-------------------------------------------------------------------------------
Test set: common.DistributedNaiveEvaluationTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.199 sec <<< FAILURE!
DistributedNaiveTest(common.DistributedNaiveEvaluationTest) Time elapsed: 0.199 sec <<< ERROR!
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure: Lost task 0.0 in stage 1.0 (TID 1, localhost): java.io.IOException: unexpected exception type
at java.base/java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1626)
at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1256)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2151)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:237)
at java.base/jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1250)
... 28 more
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
... 37 more
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1450)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1438)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1437)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1437)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:811)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:811)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1659)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1618)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1607)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:632)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1871)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1934)
at org.apache.spark.rdd.RDD$$anonfun$reduce$1.apply(RDD.scala:983)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:358)
at org.apache.spark.rdd.RDD.reduce(RDD.scala:965)
at org.apache.spark.api.java.JavaRDDLike$class.reduce(JavaRDDLike.scala:384)
at org.apache.spark.api.java.AbstractJavaRDDLike.reduce(JavaRDDLike.scala:45)
at common.NaiveEvaluation.parallelEvaluatePolynomial(NaiveEvaluation.java:46)
at common.DistributedNaiveEvaluationTest.DistributedNaiveTest(DistributedNaiveEvaluationTest.java:53)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: unexpected exception type
at java.base/java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1626)
at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1256)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2151)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2410)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2304)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2142)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1646)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:464)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.reflect.InvocationTargetException
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.lang.invoke.SerializedLambda.readResolve(SerializedLambda.java:237)
at java.base/jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1250)
... 28 more
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
... 37 more
from dizk.
SilverPoker
commented on July 30, 2024
I summarized the error information in the test as below:
common.DistributedNaiveEvaluationTest
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at common.NaiveEvaluation.$deserializeLambda$(NaiveEvaluation.java:1)
... 37 more
reductions.r1cs.R1CSRelationRDDTest
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
... 37 more
reductions.MatMulTest
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
... 37 more
reductions.R1CSConstructionTest
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
... 37 more
zk_proof_systems.zkSNARK.DistributdzkSNARKTest
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
at relations.r1cs.R1CSRelationRDD.$deserializeLambda$(R1CSRelationRDD.java:1)
... 37 more
from dizk.
howardwu
commented on July 30, 2024
@SilverPoker can you print out mvn -v? And are you compiling and running this in an IDE or terminal?
I wasn't able to reproduce the lambda deserialization errors on my machine. From an initial search online, it looks like Java lambdas are compiled differently depending on the compiler, which may have led to your issue (see here for example)
from dizk.
SilverPoker
commented on July 30, 2024
Hi, thanks for your help! I tried using mvn clean and reinstalling the compiler, it works now.
By the way, I noticed that in the paper you have described two important algorithms but didn't really implement them.
(1) For multiexp, the paper uses Pippenger’s algorithm but the code only has the implementation of Bos–Coster algorithm
(2) For the setup phase, there is one way described in the paper to avoid the unbalanced sparse and dense tasks but the code only has the implementation of Strawman. There are still task unbalanced problems...
Is my understanding correct? If there are some implementations of those two, could you please refer me to the file? Thanks!
from dizk.
howardwu
commented on July 30, 2024
I'm glad the test error was resolved.
To answer your questions:
(1) It indeed looks like the Pippenger's algorithm is missing from the library, very odd. I'll add an implementation in, thanks for catching that.
(2) The sparse and dense tasks are processed in isolation as you can see here.
from dizk.
howardwu
commented on July 30, 2024
from dizk.
howardwu
commented on July 30, 2024
As the original issue was resolved, closing this thread. Feel free to open a new one if there are further issues, thanks for digging into the codebase.
from dizk.
Related Issues (10)
- Feature request: read R1CS as serialized by libsnark HOT 4
- [Bug] Bad equality test when computing Lagrange coefficients
- LinearCombination.java evaluate method
- Rust question HOT 4
- Applications (photo proof and ML model) HOT 2
- Error in unit tests HOT 3
- Profiler should throw error for non-existent method call
- Verifier Fails in problem size 16, 17 and 18
- [Bug] Problems with Window Table
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dizk.