Skip to content

I run the apps.CifarApp,but there were some mistakes. #147

@dahaichangge

Description

@dahaichangge

I run it on the virtual machine:Centos7, hadoop2.6.0, spark1.6.0, cuda-7.5. These mistakes were encountered is shown below.
Any help would be greatly appreciated!
Many thanks in advance.

Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 12.0 failed 1 times, most recent failure: Lost task 0.0 in stage 12.0 (TID 34, localhost): java.lang.UnsatisfiedLinkError: no jnicaffe in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632)
at org.bytedeco.javacpp.Loader.load(Loader.java:470)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe.(caffe.java:16)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.bytedeco.javacpp.Loader.load(Loader.java:442)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe$NetParameter.(caffe.java:1946)
at apps.CifarApp$$anonfun$main$3.apply$mcVI$sp(CifarApp.scala:83)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp378041962097/libjnicaffe.so: /tmp/javacpp378041962097/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:615)
... 23 more

Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:910)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.foreach(RDD.scala:910)
at apps.CifarApp$.main(CifarApp.scala:82)
at apps.CifarApp.main(CifarApp.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.UnsatisfiedLinkError: no jnicaffe in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632)
at org.bytedeco.javacpp.Loader.load(Loader.java:470)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe.(caffe.java:16)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.bytedeco.javacpp.Loader.load(Loader.java:442)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe$NetParameter.(caffe.java:1946)
at apps.CifarApp$$anonfun$main$3.apply$mcVI$sp(CifarApp.scala:83)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp378041962097/libjnicaffe.so: /tmp/javacpp378041962097/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:615)
... 23 more
16/10/04 09:15:27 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.116.136:4040
16/10/04 09:15:27 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/10/04 09:15:28 INFO storage.MemoryStore: MemoryStore cleared
16/10/04 09:15:28 INFO storage.BlockManager: BlockManager stopped
16/10/04 09:15:28 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
16/10/04 09:15:28 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/10/04 09:15:28 INFO spark.SparkContext: Successfully stopped SparkContext
16/10/04 09:15:28 INFO util.ShutdownHookManager: Shutdown hook called
16/10/04 09:15:28 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-66f96f4b-4f6e-4600-9399-6caa0ac1aa46
16/10/04 09:15:28 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-66f96f4b-4f6e-4600-9399-6caa0ac1aa46/httpd-85eb2e4d-bcb9-43de-9f44-5e936dc3f9ec
[root@master bin]#

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions