I have modified the table using hiveql. Then it didn't work to show the table from spark-sql. ERROR: path does not exist
I changed the table by HiveQL.
"ALTER TABLE new_law_area_2 RENAME TO law_area"
I then intended to show my table using spark-sql.
"SELECT * FROM law_area LIMIT 10"
But it didn't work ... with this error.
org.spark_project.guava.util.concurrent.UncheckedExecutionException:org.apache.spark.sql.AnalysisException
17/04/18 14:17:47 ERROR SparkSQLDriver: Failed in [select * from law_area limit 10] org.spark_project.guava.util.concurrent.UncheckedExecutionException: org.apache.spark.sql.AnalysisException: Path does not exist: hdfs: //dmlab/apps/hive/warehouse/dimension.db/new_law_area_2; at org.spark_project.guava.cache.LocalCache $ LocalLoadingCache.getUnchecked (LocalCache.java:4882) at org.spark_project.guava.cache.LocalCache $ LocalLoadingCache.apply (LocalCache.java:4898) at org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation (HiveMetastoreCatalog.scala: 128) at org.apache.spark.sql.hive.HiveSessionCatalog.lookupRelation (HiveSessionCatalog.scala: 70) at org.apache.spark.sql.catalyst.analysis.Analyzer $ ResolveRelations $ .org $ apache $ spark $ sql $ catalyst $ analysis $ Analyzer $ ResolveRelations $$ lookupTableFromCatalog (Analyzer.scala: 457) at org.apache.spark.sql.catalyst.analysis.Analyzer $ ResolveRelations $$ anonfun $ apply $ 8.applyOrElse (Analyzer.scala: 479) at org.apache.spark.sql.catalyst.analysis.Analyzer $ ResolveRelations $$ anonfun $ apply $ 8.applyOrElse (Analyzer.scala: 464) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ resolveOperators $ 1.apply (LogicalPlan.scala: 61) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ resolveOperators $ 1.apply (LogicalPlan.scala: 61) at org.apache.spark.sql.catalyst.trees.CurrentOrigin $ .withOrigin (TreeNode.scala: 70) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperators (LogicalPlan.scala: 60) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.trees.TreeNode $$ anonfun $ 4.apply (TreeNode.scala: 307) at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator (TreeNode.scala: 188) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren (TreeNode.scala: 305) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperators (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.trees.TreeNode $$ anonfun $ 4.apply (TreeNode.scala: 307) at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator (TreeNode.scala: 188) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren (TreeNode.scala: 305) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperators (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan $$ anonfun $ 1.apply (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.trees.TreeNode $$ anonfun $ 4.apply (TreeNode.scala: 307) at org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator (TreeNode.scala: 188) at org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren (TreeNode.scala: 305) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperators (LogicalPlan.scala: 58) at org.apache.spark.sql.catalyst.analysis.Analyzer $ ResolveRelations $ .apply (Analyzer.scala: 464) at org.apache.spark.sql.catalyst.analysis.Analyzer $ ResolveRelations $ .apply (Analyzer.scala: 454) at org.apache.spark.sql.catalyst.rules.RuleExecutor $$ anonfun $ execute $ 1 $$ anonfun $ apply $ 1.apply (RuleExecutor.scala: 85) at org.apache.spark.sql.catalyst.rules.RuleExecutor $$ anonfun $ execute $ 1 $$ anonfun $ apply $ 1.apply (RuleExecutor.scala: 82) at scala.collection.LinearSeqOptimized $ class.foldLeft (LinearSeqOptimized.scala: 124) at scala.collection.immutable.List.foldLeft (List.scala: 84) at org.apache.spark.sql.catalyst.rules.RuleExecutor $$ anonfun $ execute $ 1.apply (RuleExecutor.scala: 82) at org.apache.spark.sql.catalyst.rules.RuleExecutor $$ anonfun $ execute $ 1.apply (RuleExecutor.scala: 74) at scala.collection.immutable.List.foreach (List.scala: 381) at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute (RuleExecutor.scala: 74) at org.apache.spark.sql.execution.QueryExecution.analyzed $ lzycompute (QueryExecution.scala: 69) at org.apache.spark.sql.execution.QueryExecution.analyzed (QueryExecution.scala: 67) at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed (QueryExecution.scala: 50) at org.apache.spark.sql.Dataset $ .ofRows (Dataset.scala: 63) at org.apache.spark.sql.SparkSession.sql (SparkSession.scala: 592) at org.apache.spark.sql.SQLContext.sql (SQLContext.scala: 699) at org.apache.spark.sql.hive.thriftserver.SparkSQLDriver.run (SparkSQLDriver.scala: 62) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd (SparkSQLCLIDriver.scala: 335) at org.apache.hadoop.hive.cli.CliDriver.processLine (CliDriver.java:376) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver $ .main (SparkSQLCLIDriver.scala: 247) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main (SparkSQLCLIDriver.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:497) at org.apache.spark.deploy.SparkSubmit $ .org $ apache $ spark $ deploy $ SparkSubmit $$ runMain (SparkSubmit.scala: 745) at org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1 (SparkSubmit.scala: 187) at org.apache.spark.deploy.SparkSubmit $ .submit (SparkSubmit.scala: 212) at org.apache.spark.deploy.SparkSubmit $ .main (SparkSubmit.scala: 126) at org.apache.spark.deploy.SparkSubmit.main (SparkSubmit.scala)
How can I solve this problem?
plz ... my all tables didn't work this way ...
I want to use spark-sql
+3
source to share
2 answers