What is the difference between spark vs pyspark?

In the context of big data analysis or big data Hadoop, the difference between Spark vs Pyspark is minor. Spark is mainly used for the overall framework, on the other hand, the pyspark specifically provides the Python API for Spark. Therefore, the pyspark allows the Python programming language to interact with the spark functionalities well. PySpark generally is a Python library that enables seamless integration with spark distributed computing capabilities.

Here is the basic example given to showcase the difference between Spark and Pyspark:-

Using spark in scala:-

// Spark code in Scala
Val textFile = sc.textFile(“hdfs://path/to/your/file.txt”)
Val counts = textFile.flatMap(line => line. split(“ “))
                     .map(word => (word, 1))
                     .reduceByKey(_ + _)
Counts.saveAsTextFile(“hdfs://path/to/save/wordCountOutput”)
Using Pyspark in Python:- 
# PySpark code in Python
From pyspark import SparkContext
Sc = SparkContext(appName=” WordCount”)
textFile = sc.textFile(“hdfs://path/to/your/file.txt”)
counts = textFile.flatMap(lambda line: line. split(“ “)) 
                 .map(lambda word: (word, 1)) 
                 .reduceByKey(lambda a, b: a + b)
Counts.saveAsTextFile(“hdfs://path/to/save/wordCountOutput”)