I am new to Spark.
While using pyspark, when I attempt to initialize a new SparkContext,
from pyspark import SparkContext
sc = SparkContext("local[4]", "test")
I get the following error:
ValueError: Cannot run multiple SparkContexts at onceI'm wondering if my previous attempts at running example code loaded something into memory that didn't clear out.
Log in to share your answer and help other learners.
Log in to answerBest Answer · By JanBask Salesforce Expert
Answered on Jul 12, 2021
To solve cannot run multiple sparkcontexts at once
running ./bin/pyspark interactively automatically loads the sparkContext.
You will even see that written on the screen when you execute pyspark.
So, you can either run "del sc" or stop.sc() at the beginning and create a new sparkcontext:
→ stop.sc() → sc = SparkContext.getOrCreate()
or just carry on and use "sc" as automatically defined.
Free tutorials and interview questions from industry experts — learn the skill, then get ready to prove it.
Step-by-step Salesforce guides from industry experts
Common Salesforce interview questions, answered
Guides, tips and career advice on Salesforce from JanBask experts.
Salesforce Cost for Salesforce Certification: Exam Fees by Type, Retakes & Hidden Costs
Salesforce certification exam fees run $75 to $4,500, and most cost $200. See cost by role, retakes, taxes, discounts, and…
Salesforce Salesforce Admin Certification Cost 2026: Exam Fees, Retake & Other Expenses
Learn the Salesforce Admin certification cost in 2026, including the exam fee, retake cost, taxes, discounts, preparation…
Salesforce How to Get Into Salesforce With No Experience
Wondering if you can get a Salesforce job with no experience? Learn how to pick a career path, build real skills, and land your…
Salesforce How to Become a Salesforce Developer Without Experience
No coding background? Learn how to become a Salesforce Developer step by step - skills to build, projects to create, and how to…