Skip to content

README.md

Latest commit

 

History

History
46 lines (30 loc) · 555 Bytes

README.md

File metadata and controls

46 lines (30 loc) · 555 Bytes

Very simple Spark interact to Kafka

Setup

  1. Create python environment

Create python virtual environment and prepare data

python3 -m venv ./venv
source ./venv/bin/activate

Install package

pip3 install -r requirements.txt

Except all sh file

chmod +x script/*.sh

Download NASA logging data

./script/prepare.sh
  1. Run kafka cluster written by docker
docker-compose up -d
  1. Run simple demo spark
./script/run.sh