Flink安装

发布于 2025年9月28日 06:24:41 (武汉时间)

Flink安装

虚拟机环境：
Ubuntu 22.04.2
Java 11

1.1 安装

  1. cd /usr/local
  2. 下载安装包
sudo wget https://mirrors.aliyun.com/apache/flink/flink-1.17.2/flink-1.17.2-bin-scala_2.12.tgz

sudo tar -zxvf flink-1.17.2-bin-scala_2.12.tgz

vim ~/.bashrc
底部黏贴以下代码：
export FLINK_HOME=/usr/local/flink-1.17.2
export PATH=$PATH:$FLINK_HOME/bin
保存更改：
source ~/.bashrc

修改权限：
sudo chown -R parallels:parallels /usr/local/flink-1.17.2
注意：这个parallels:parallels是你的虚拟机当前用户的名字

vim /usr/local/flink-1.17.2/conf/flink-conf.yaml

在底部加入:
# 并行度（单机可设为 CPU 核数）
parallelism.default: 4

将taskmanager.numberOfTaskSlots: 1
改成：taskmanager.numberOfTaskSlots: 4

1.2 测试执行

  1. 进入/usr/local/flink-1.17.2/bin目录中
./start-cluster.sh

以下即为成功启动：

parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2/bin$ ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host ubuntu-linux-22-04-02-desktop.
Starting taskexecutor daemon on host ubuntu-linux-22-04-02-desktop.
parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2/bin$ jps
9510 StandaloneSessionClusterEntrypoint
9805 TaskManagerRunner
9967 Jps

1.2.3 测试样例：

  1. cd .. 进入/usr/local/flink-1.17.2目录
./bin/flink run -p 1 ./examples/streaming/WordCount.jar

以下是运行结果：

parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2$ ./bin/flink run -p 1 ./examples/streaming/WordCount.jar
Executing example with default input data.
Use --input to specify file input.
Printing result to stdout. Use --output to specify output path.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.flink.api.java.ClosureCleaner (file:/usr/local/flink-1.17.2/lib/flink-dist-1.17.2.jar) to field java.lang.String.value
WARNING: Please consider reporting this to the maintainers of org.apache.flink.api.java.ClosureCleaner
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Job has been submitted with JobID 867c7b8829875c5797f76b80e9320f4c
Program execution finished
Job with JobID 867c7b8829875c5797f76b80e9320f4c has finished.
Job Runtime: 745 ms

parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2$ tail -f log/flink-*-taskexecutor-*.out
(nymph,1)
(in,3)
(thy,1)
(orisons,1)
(be,4)
(all,2)
(my,1)
(sins,1)
(remember,1)
(d,4)

退出：CTRL+C

然后关闭flink：

 ./bin/stop-cluster.sh

结果：

parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2$ ./bin/stop-cluster.sh
Stopping taskexecutor daemon (pid: 16605) on host ubuntu-linux-22-04-02-desktop.
Stopping standalonesession daemon (pid: 16305) on host ubuntu-linux-22-04-02-desktop.
parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2$ jps
18374 Jps
parallels@ubuntu-linux-22-04-02-desktop:/usr/local/flink-1.17.2$

2.1 一键安装脚本

创建文件：vim install-flink.sh
加权：chmod +x install-flink.sh
运行：./install-flink.sh

#!/bin/bash

# install-flink.sh - 一键安装 Flink 1.17.2 单机版（Ubuntu 22.04 + Java 11）
# 支持下载进度显示，避免“假死”等待

set -e

FLINK_VERSION="1.17.2"
FLINK_DIR="/usr/local/flink-${FLINK_VERSION}"
USER_NAME=$(whoami)
DOWNLOAD_URL="https://mirrors.aliyun.com/apache/flink/flink-${FLINK_VERSION}/flink-${FLINK_VERSION}-bin-scala_2.12.tgz"
TAR_FILE="flink-${FLINK_VERSION}.tgz"

echo "🚀 开始安装 Apache Flink ${FLINK_VERSION} ..."
echo "👤 当前用户: ${USER_NAME}"
echo "📁 安装路径: ${FLINK_DIR}"

# 1. 检查 Java
if ! command -v java &> /dev/null; then
    echo "❌ 错误: 未找到 Java，请先安装 Java 11"
    exit 1
fi

JAVA_VERSION=$(java -version 2>&1 | head -1 | cut -d'"' -f2 | cut -d'.' -f1)
if [ "$JAVA_VERSION" != "11" ]; then
    echo "⚠️  警告: 检测到 Java 版本为 $JAVA_VERSION，Flink 1.17 推荐使用 Java 11"
fi

# 2. 进入 /usr/local
cd /usr/local

# 3. 下载并解压（如果尚未存在）
if [ ! -d "$FLINK_DIR" ]; then
    if [ -f "$TAR_FILE" ]; then
        echo "🗑️  删除已存在的旧压缩包: $TAR_FILE"
        sudo rm -f "$TAR_FILE"
    fi

    echo "📥 正在从阿里云镜像下载 Flink ${FLINK_VERSION} ..."
    echo "   URL: $DOWNLOAD_URL"
    echo "   （显示下载进度和速度，请稍候...）"

    # 使用 wget 显示进度（不静默），输出到终端
    # --show-progress 需要 wget 1.16+（Ubuntu 22.04 默认满足）
    if wget --version | grep -q "GNU Wget"; then
        sudo wget --show-progress --progress=bar:force:noscroll "$DOWNLOAD_URL" -O "$TAR_FILE"
    else
        # 降级方案：普通 wget
        sudo wget "$DOWNLOAD_URL" -O "$TAR_FILE"
    fi

    echo ""
    echo "✅ 下载完成: $TAR_FILE"

    echo "📦 正在解压..."
    sudo tar -zxf "$TAR_FILE"
    sudo rm -f "$TAR_FILE"
    echo "✅ 解压完成"
else
    echo "✅ Flink 已存在，跳过下载与解压"
fi

# 4. 设置权限
echo "🔑 设置目录权限给用户: $USER_NAME"
sudo chown -R "${USER_NAME}:${USER_NAME}" "$FLINK_DIR"

# 5. 配置环境变量（仅对当前用户）
if ! grep -q "FLINK_HOME" ~/.bashrc; then
    echo "⚙️  配置环境变量..."
    echo "export FLINK_HOME=${FLINK_DIR}" >> ~/.bashrc
    echo 'export PATH=$PATH:$FLINK_HOME/bin' >> ~/.bashrc
    export FLINK_HOME="$FLINK_DIR"
    export PATH="$PATH:$FLINK_HOME/bin"
else
    echo "✅ 环境变量已配置"
    source ~/.bashrc
fi

# 6. 修改 flink-conf.yaml
CONF_FILE="${FLINK_DIR}/conf/flink-conf.yaml"

# 确保配置项存在或更新
update_config() {
    local key="$1"
    local value="$2"
    if grep -q "^$key:" "$CONF_FILE"; then
        sed -i "s|^$key:.*|$key: $value|" "$CONF_FILE"
        echo "   更新 $key -> $value"
    else
        echo "$key: $value" >> "$CONF_FILE"
        echo "   新增 $key -> $value"
    fi
}

echo "🔧 正在配置 Flink 参数..."
update_config "taskmanager.numberOfTaskSlots" "4"
update_config "parallelism.default" "4"

echo "✅ Flink 配置已更新"

# 7. 询问是否启动并运行示例
echo ""
read -p "是否立即启动 Flink 并运行 WordCount 示例? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
    echo "▶️  启动 Flink 集群..."
    "$FLINK_DIR/bin/start-cluster.sh"

    echo "⏳ 等待 5 秒让集群就绪..."
    sleep 5

    echo "🧪 提交 WordCount 示例作业..."
    if "$FLINK_DIR/bin/flink" run -p 1 "$FLINK_DIR/examples/streaming/WordCount.jar"; then
        echo "✅ 作业提交成功！查看实时输出（按 Ctrl+C 退出）:"
        tail -f "$FLINK_DIR"/log/flink-*taskexecutor*.out
    else
        echo "❌ 作业提交失败，请检查日志"
        exit 1
    fi
else
    echo ""
    echo "✅ 安装完成！你可以手动操作："
    echo "   cd $FLINK_DIR"
    echo "   ./bin/start-cluster.sh"
    echo "   ./bin/flink run -p 1 ./examples/streaming/WordCount.jar"
    echo "   ./bin/stop-cluster.sh"
fi
← 返回首页
↩