↧
Answer by Jaskaran Singh for While writing to hdfs path getting error...
The error is that you are trying to write the dataframe onto the same location for each the item in your givenItemList collection. Usually if do that it should give error OutputDirectory already...
View ArticleAnswer by Alfilercio for While writing to hdfs path getting error...
You can do all the selects in one single job, get all the selects and union in a single table.Dataset<Row> resultDs = givenItemList.parallelStream().map( item -> { String query = "select $item...
View ArticleWhile writing to hdfs path getting error java.io.IOException: Failed to rename
I am using spark-sql-2.4.1v which is using hadoop-2.6.5.jar version . I need to save my data first on hdfs and move to cassandra later.Hence I am trying to save the data on hdfs as below:String...
View Article