当前位置:网站首页>Oozie integrated shell
Oozie integrated shell
2022-07-19 02:05:00 【sun_ xo】
1) prepare shell case
$ cd ~/work/oozie-5.2.1
$ tree oozie/apps/shell
oozie/apps/shell
├── hello.sh
├── job.properties
└── workflow.xml$ cat oozie/apps/shell/hello.sh
#!/bin/sh
echo "my_output=Hello Oozie"$ cat oozie/apps/shell/job.properties
nameNode=hdfs://localhost:9000
resourceManager=localhost:8032
queueName=default
oozieRoot=user/${user.name}/oozie
oozie.wf.application.path=${nameNode}/${oozieRoot}/apps/shell
EXEC=hello.sh$ cat oozie/apps/shell/workflow.xml
<workflow-app xmlns="uri:oozie:workflow:1.0" name="shell-wf">
<start to="shell-node"/>
<action name="shell-node">
<shell xmlns="uri:oozie:shell-action:1.0">
<resource-manager>${resourceManager}</resource-manager>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>${EXEC}</exec>
<file>${nameNode}/${oozieRoot}/apps/shell/${EXEC}#${EXEC}</file>
<capture-output/>
</shell>
<ok to="check-output"/>
<error to="fail"/>
</action>
<decision name="check-output">
<switch>
<case to="end">
${wf:actionData('shell-node')['my_output'] eq 'Hello Oozie'}
</case>
<default to="fail-output"/>
</switch>
</decision>
<kill name="fail">
<message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<kill name="fail-output">
<message>Incorrect output, expected [Hello Oozie] but was [${wf:actionData('shell-node')['my_output']}]</message>
</kill>
<end name="end"/>
</workflow-app>2) upload to hdfs
$ hdfs dfs -put oozie/apps/shell oozie/apps/
3) run and check
$ bin/oozie job -config oozie/apps/shell/job.properties -run
job: 0000004-220629164930563-oozie-sun_-W
$ bin/oozie job -info 0000004-220629164930563-oozie-sun_-W
Job ID : 0000004-220629164930563-oozie-sun_-W
------------------------------------------------------------------------------------------------------------------------------------
Workflow Name : shell-wf
App Path : hdfs://localhost:9000/user/sun_xo/oozie/apps/shell
Status : SUCCEEDED
Run : 0
User : sun_xo
Group : -
Created : 2022-06-29 09:22 GMT
Started : 2022-06-29 09:22 GMT
Last Modified : 2022-06-29 09:22 GMT
Ended : 2022-06-29 09:22 GMT
CoordAction ID: -
Actions
------------------------------------------------------------------------------------------------------------------------------------
ID Status Ext ID Ext Status Err Code
------------------------------------------------------------------------------------------------------------------------------------
[email protected]:start: OK - OK -
------------------------------------------------------------------------------------------------------------------------------------
[email protected] OK application_1656492227290_0007SUCCEEDED -
------------------------------------------------------------------------------------------------------------------------------------
[email protected] OK - end -
------------------------------------------------------------------------------------------------------------------------------------
[email protected] OK - OK -
------------------------------------------------------------------------------------------------------------------------------------
Note:
Shell action’s stdout and stderr output are redirected to the Oozie Launcher map-reduce job task STDOUT that runs the shell command
The output can be found under hdfs://localhost:9000/tmp/sun_xo/logs/application_1656492227290_0007/
application_1656492227290_0007 is ext_id of shell-node
边栏推荐
- 动手学深度学习---深度学习计算篇
- VS Code 问题:launch:program‘...\.vscode\launch.exe‘ dose not exist
- Mxnet network model (IV) Gan neural network
- Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination
- Hands on deep learning -- from full connection layer to convolution layer
- 01基于RFID的智能仓储管理系统设计
- The code of dsaa related articles in the blog
- 霍夫变换讲解
- Fisher线性判别分析Fisher Linear Distrimination
- Boost thread pool
猜你喜欢
![[MySQL] windows install MySQL 5.7](/img/71/be5b0cc3e130c2b9f3884d90b9cd39.jpg)
[MySQL] windows install MySQL 5.7

Labelme 的简单用法和界面介绍

CAN协议通信

关于1000BASE-T1 1000BASE-TX和100BASE-T1

【白话模电1】PN结与二极管

Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination

Fair Multiple Decision Making Through Soft Interventions

ROC 曲线讲解 (Receiver Operarating Curve)

YYDS!阿里技术官最新总结的分布式核心技术笔记已上线,堪称福音

VIM profile
随机推荐
L1,L2范数
[literature reading] mcunet: tiny deep learning on IOT devices
[cute new problem solving] sum of three numbers
VIM 配置文件
动手学深度学习---深度学习计算篇
中心极限定理
VGG (Visual Geometry Group)
偏差(bias)和方差(variance)
Boost thread pool
06 design of smart electronic medicine box based on stm32
Nacos配置管理
FS32K148调试之WDOG与电源模式
【pycharm】Cannot find reference ‘XXX‘ in ‘__ init__. Py 'solution
Oozie 集成 Shell
Basic principle and parameter interpretation of operational amplifier
SABER 最强大的数模混合信号仿真软件
03基于ZigBee的城市道路除尘降温系统设计
Allegro design entry CIS and OrCAD capture CIS relationship
电解电容特性及应用要点
[go language] code coverage test (Gcov)