PySpark V3.0 conda environment is introduced.
- Services: Data Science
- Release Date: June 01, 2021
Use the PySpark V3.0 conda to create Data Flow jobs or run PySpark locally. The PySpark version is updated from V2.4.4 to V3.0.2 in this conda environment to be compatible with Data Flow upgrades. The conda is based on Python 3.7 with the Oracle Accelerated Data Science (ADS) SDK v2.2.1 library. It provides support for working with the Oracle Autonomous Database and snappy compression in parquet files. This conda environment is for CPUs. The slug name is pyspark30_p37_cpu_v1.
For more information, see Data Science and Data Science API.