Spark and Hive integration

Spark Sql and Hql

[cloudera@quickstart ~]$ sudo find / -name 

'hive-site.xml'

[cloudera@quickstart ~]$ sudo chmod -R 777 

/usr/lib/spark/conf
[cloudera@quickstart ~]$ cp 

/etc/hive/conf.dist/hive-site.xml 

/usr/lib/spark/conf
_____________________________________
from hive-site.xml -->  

set hive.metastore.warehouse.dir to some directory path

import org.apache.spark.sql.hive.HiveContext

 val hq = new HiveContext(sc);

    

    val hivedf = hq.sql("use leela_db")

    val hivedf2 = hq.sql("select * from emp_sal")

    hivedf2.show();

    val sal_avg = hq.sql("select avg(salary) from emp_sal")

    sal_avg.show()