AnsweredAssumed Answered

Dynamically pass a value to the hive query

Question asked by Mrinmoy_Das_8405 on Sep 7, 2016
Latest reply on Sep 7, 2016 by Mrinmoy_Das_8405
I am very new to UC4. I am using UC4 for scheduling the job for my BIG Data ingestion project where I am using Hive. My requirement is dynamically I need to find the next available partition date of 1 hive table and then pass that date to a hive query. I have done the below steps.

I have used  shell job and inside process tab I have written

partitions=$(hive -e "SHOW PARTITIONS default.idmap_snapshot")
do
for partition in $partitions
do
partition_date=`echo "$partition" | cut -c4-11`
if [ $partition_date -gt 20160718 ]; then
echo $partition_date
break
fi  
done

hive -e "/dw/etl/home/dev/hql/dw_apt/dw_apt.dw_merged_profile_w_dtl_adelphic_us.ins.hql $partition_date"

Please let me know whether my approach is correct or not? Right now I am facing some access issue for which I cannot debug the job. Any help would be highly appreciated.

Thanks,
Mrinmoy 

Outcomes