0

Estoy intentando hacer el "import org.apache.spark.SparkConf" en IntelliJ, con Scala y me está devolviendo el siguiente error:

Error while importing sbt project:

[info] Loading project definition from C:\Users\aresa\IdeaProjects\First_Project_Spark_Scala\project
[info] Loading settings for project first_project_spark_scala from build.sbt ...
[info] Set current project to First_Project_Spark_Scala (in build file:/C:/Users/aresa/IdeaProjects/First_Project_Spark_Scala/)
[info] sbt server started at local:sbt-server-6cc4be77775cbe870033
sbt:First_Project_Spark_Scala>
error: error while loading String, class file '/modules/java.base/java/lang/String.class' is broken
(class java.lang.NullPointerException/null)
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepFiles
[info]  Run `last` for details.
[info] Reapplying settings...
[info] Set current project to First_Project_Spark_Scala (in build file:/C:/Users/aresa/IdeaProjects/First_Project_Spark_Scala/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from C:/Users/aresa/.IdeaIC2019.1/config/plugins/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1+4-88400d3f/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] Set current project to First_Project_Spark_Scala (in build file:/C:/Users/aresa/IdeaProjects/First_Project_Spark_Scala/)
[info] Updating ...
[warn]  module not found: org.apache.spark#spark-core_2.13;2.13.0
[warn] ==== local: tried
[warn]   C:\Users\aresa\.ivy2\local\org.apache.spark\spark-core_2.13\2.13.0\ivys\ivy.xml
[warn] ==== public: tried
[warn]   https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.13/2.13.0/spark-core_2.13-2.13.0.pom
[warn] ==== local-preloaded-ivy: tried
[warn]   C:\Users\aresa\.sbt\preloaded\org.apache.spark\spark-core_2.13\2.13.0\ivys\ivy.xml
[warn] ==== local-preloaded: tried
[warn]   file:/C:/Users/aresa/.sbt/preloaded/org/apache/spark/spark-core_2.13/2.13.0/spark-core_2.13-2.13.0.pom
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.apache.spark#spark-core_2.13;2.13.0: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Unresolved dependencies path:
[warn]      org.apache.spark:spark-core_2.13:2.13.0 (C:\Users\aresa\IdeaProjects\First_Project_Spark_Scala\build.sbt#L7-8)
[warn]        +- default:first_project_spark_scala_2.13:0.1
[error] sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.13;2.13.0: not found
[error]     at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error]     at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error]     at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error]     at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error]     at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error]     at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error]     at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error]     at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error]     at xsbt.boot.Using$.withResource(Using.scala:10)
[error]     at xsbt.boot.Using$.apply(Using.scala:9)
[error]     at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error]     at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error]     at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error]     at xsbt.boot.Locks$.apply(Locks.scala:28)
[error]     at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error]     at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error]     at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error]     at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error]     at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error]     at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error]     at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error]     at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error]     at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error]     at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2561)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error]     at sbt.std.Transform$$anon$4.work(System.scala:67)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]     at sbt.Execute.work(Execute.scala:278)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]     at java.base/java.lang.Thread.run(Thread.java:835)
[error] sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.13;2.13.0: not found
[error]     at sbt.internal.librarymanagement.IvyActions$.resolveAndRetrieve(IvyActions.scala:332)
[error]     at sbt.internal.librarymanagement.IvyActions$.$anonfun$updateEither$1(IvyActions.scala:208)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.$anonfun$withModule$1(Ivy.scala:239)
[error]     at sbt.internal.librarymanagement.IvySbt.$anonfun$withIvy$1(Ivy.scala:204)
[error]     at sbt.internal.librarymanagement.IvySbt.sbt$internal$librarymanagement$IvySbt$$action$1(Ivy.scala:70)
[error]     at sbt.internal.librarymanagement.IvySbt$$anon$3.call(Ivy.scala:77)
[error]     at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:95)
[error]     at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:80)
[error]     at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:99)
[error]     at xsbt.boot.Using$.withResource(Using.scala:10)
[error]     at xsbt.boot.Using$.apply(Using.scala:9)
[error]     at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:60)
[error]     at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:50)
[error]     at xsbt.boot.Locks$.apply0(Locks.scala:31)
[error]     at xsbt.boot.Locks$.apply(Locks.scala:28)
[error]     at sbt.internal.librarymanagement.IvySbt.withDefaultLogger(Ivy.scala:77)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:199)
[error]     at sbt.internal.librarymanagement.IvySbt.withIvy(Ivy.scala:196)
[error]     at sbt.internal.librarymanagement.IvySbt$Module.withModule(Ivy.scala:238)
[error]     at sbt.internal.librarymanagement.IvyActions$.updateEither(IvyActions.scala:193)
[error]     at sbt.librarymanagement.ivy.IvyDependencyResolution.update(IvyDependencyResolution.scala:20)
[error]     at sbt.librarymanagement.DependencyResolution.update(DependencyResolution.scala:56)
[error]     at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:45)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:93)
[error]     at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:68)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$19(LibraryManagement.scala:106)
[error]     at scala.util.control.Exception$Catch.apply(Exception.scala:224)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:106)
[error]     at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:89)
[error]     at sbt.util.Tracked$.$anonfun$inputChanged$1(Tracked.scala:149)
[error]     at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:120)
[error]     at sbt.Classpaths$.$anonfun$updateTask$5(Defaults.scala:2561)
[error]     at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error]     at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error]     at sbt.std.Transform$$anon$4.work(System.scala:67)
[error]     at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error]     at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]     at sbt.Execute.work(Execute.scala:278)
[error]     at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error]     at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error]     at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error]     at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]     at java.base/java.lang.Thread.run(Thread.java:835)
[error] (update) sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.13;2.13.0: not found
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.13;2.13.0: not found
[error] Total time: 1 s, completed 21 jul. 2019 20:31:42
[info] shutting down server

La cuestión es que no sé si me he dejado de instalar algún plugin o algo. No logro encontrar la solución del problema.

En un principio he instalado Scala y sbt, y a continuación java version 12. Una vez creado el proyecto he creado un objeto y ahí es donde he creado la función "main" con el import ya comentado.

package main

import org.apache.spark.SparkConf

object PrimerProyecto {
  def main(args: Array[String]) {

El contenido del build.sbt es el siguiente:


version := "0.1"

scalaVersion := "2.11.8"

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-sql_2.11" % "2.0.0" % "provided",
  "org.apache.spark" % "spark-core_2.11" % "2.0.0" % "provided")

Si es necesario aportar más detalles del error, los daré encantado.

Muchas gracias!

1 respuesta 1

0

Yo cree un proyecto SBt en IntelliJ con las siguientes caracteristicas y compilo correctamente:

Build.sbt

version := "0.1"

scalaVersion := "2.13.0"

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core_2.11" % "2.0.0" % "provided",
  "org.apache.spark" % "spark-sql_2.11" % "2.0.0" % "provided"   
)

Primer Proyecto:

package main

import org.apache.spark.SparkConf

object PrimerProyecto {

  object PrimerProyecto {
    def main(args: Array[String]) {
    }
  }   
}

Tu Respuesta

By clicking “Publica tu respuesta”, you agree to our terms of service and acknowledge you have read our privacy policy.

¿No es la respuesta que buscas? Examina otras preguntas con la etiqueta o formula tu propia pregunta.