Giter Club home page Giter Club logo

top-k-route-with-spark's People

Contributors

kim-taesu avatar

Watchers

avatar avatar

top-k-route-with-spark's Issues

TaskMemoryManager: Failed to allocate a page(... bytes), try again 오류

dest size를 15로 설정하고 Top K 코드 실행 시 WARN 오류 찍힘

20/07/20 18:36:52 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:53 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.
20/07/20 18:36:54 WARN TaskMemoryManager: Failed to allocate a page (536870912 bytes), try again.

Window에서 spark-submit 명령 수행 시 Exception while deleting Spark temp dir 오류 

java.io.IOException: Failed to delete: C:\Users\BLAHBLAHBLAH\AppData\Local\Temp\spark-87a0b959-9d1c-4416-989f-b5691303e28b\userFiles-92fb9c90-a9b8-4605-b3dc-23aced2dda8e\top-k-jar-with-dependencies.jar
        at org.apache.spark.network.util.JavaUtils.deleteRecursivelyUsingJavaIO(JavaUtils.java:144)
        at org.apache.spark.network.util.JavaUtils.deleteRecursively(JavaUtils.java:118)
        at org.apache.spark.network.util.JavaUtils.deleteRecursivelyUsingJavaIO(JavaUtils.java:128)
        at org.apache.spark.network.util.JavaUtils.deleteRecursively(JavaUtils.java:118)
        at org.apache.spark.network.util.JavaUtils.deleteRecursively(JavaUtils.java:91)
        at org.apache.spark.util.Utils$.deleteRecursively(Utils.scala:1062)
        at org.apache.spark.SparkEnv.stop(SparkEnv.scala:103)
        at org.apache.spark.SparkContext$$anonfun$stop$11.apply$mcV$sp(SparkContext.scala:1974)
        at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1340)
        at org.apache.spark.SparkContext.stop(SparkContext.scala:1973)
        at org.apache.spark.SparkContext$$anonfun$2.apply$mcV$sp(SparkContext.scala:575)
        at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:216)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:188)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:188)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:188)
        at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1945)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:188)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:188)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:188)
        at scala.util.Try$.apply(Try.scala:192)
        at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)
        at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.