
回复
只需几个简单的步骤即可启动并运行Flink示例程序。
唯一要求是装有Java 8,检查Java正确安装:
直接下载二进制包到本地并解压。
jobmanager.rpc.address: 10.0.0.1 配置主节点的ip
jobmanager 主节点 taskmanager 从节点
vim ~/.bash_profile
# Flink
export FLINK_HOME=/Users/javaedge/Downloads/soft/flink-1.17.0
export PATH=$FLINK_HOME/bin:$PATH
source ~/.bash_profile
javaedge@JavaEdgedeMac-mini flink-1.17.0 % cd bin
javaedge@JavaEdgedeMac-mini bin % ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host JavaEdgedeMac-mini.local.
Starting taskexecutor daemon on host JavaEdgedeMac-mini.local.
javaedge@JavaEdgedeMac-mini bin % jps
验证集群启动成功:
先启动一个 socket 传输:
javaedge@JavaEdgedeMac-mini data % nc -lk 9527
javaedge
666
888
再提交任务:
./flink run -c org.apache.flink.streaming.examples.socket.SocketWindowWordCount ../examples/streaming/SocketwindowWordCount.jar --hostname localhost --port 9527
打开控制台,可见有个运行中任务了:
任务执行结果:
任务执行时,将一个任务划分为多个并行子任务来执行的能力。
使用 ExecutionEnvironment
或 StreamExecutionEnvironment
对象设置并行度,这会影响到该环境中所有算子的并行度。
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(4);
直接在算子上设置并行度,这会覆盖全局设置的并行度。
final DataStream<String> input = env.addSource(new FlinkKafkaConsumer010<>("topic", new SimpleStringSchema(), props));
input.flatMap(new MyFlatMapFunction()).setParallelism(2).print();
并行度设置要根据具体场景和资源而调整:
./flink run -c org.apache.flink.streaming.examples.socket.SocketWindowWordCount -p 2 ../examples/streaming/SocketwindowWordCount.jar --hostname localhost --port 9527
参考
文章转载自公众号: JavaEdge