Skip to content

30-hour plan ​

This plan assumes you read Java comfortably, have never written Python, and have roughly 64 hours between now and check-in. Adjust the numbers to your own window.

Time budget ​

ItemHours
Window until check-inabout 64
Sleep (3 nights × 8 h)−24
Awake timeabout 40
Meals and breaksabout −8
Usable study timeabout 32
Plan30 (28.5 of blocks + 1.5 buffer)

Principles ​

  1. Spark first. Everything here is chosen because the sprint needs it. Python you will never meet in Spark can wait.
  2. Type the code. Reading is not learning. Every block ends with something you ran.
  3. Start the build before you study. PySpark tests need Spark compiled first, and that takes a while.
  4. unittest before pytest. PySpark tests are classes with self, in the unittest style, run through python/run-tests.
  5. If time runs short, cut in this order: decorators and generators (Block 2), then the optional reading in Block 3. Never cut the environment, typing or coverage.

Block 0: accounts and build (0.5 h, tonight, first) ​

  • ☐ Request an Apache JIRA account (form).
  • ☐ Fork apache/spark on GitHub.
  • ☐ In your fork, open the Actions tab and enable the "Build and test" and "Report test results" workflows.
  • ☐ Clone your fork and add upstream.
  • ☐ Start the Maven or sbt build in a terminal and leave it running (commands in the setup checklist).

Done when: the build is running and you have submitted the JIRA request.

Block 1: Python syntax (7 h) ​

TimeTask
1.5 hLearn X in Y Minutes: Python, typing each snippet into a REPL
3.5 hThe exercises of the PyLadies workshop curriculum. Write every exercise yourself, even the easy ones
1 hW3Schools Python only to fill gaps: slicing, dict, set, comprehensions, f-strings, exceptions, imports
1 hSet up a virtual environment (venv or uv) and run a script from the terminal

Done when: you can write a 40-line script with functions, a dict, a list comprehension and a try/except without looking anything up. Read Python for Java developers alongside this block.

Block 2: Python you will meet in real code (6 h) ​

TimeTask
3 hClasses, __init__, dunder methods, @dataclass, @property, decorators, generators, with, *args and **kwargs
2 hunittest.TestCase: setUp, assertEqual, assertRaises, subTest, mock.patch
0.5 hbreakpoint() and the pdb basics
0.5 hPort a small Java class you know (with its test) to Python

Done when: you can read a test file in the Spark repository and explain what each line does.

Block 3: typing and mypy (5 h) ​

TimeTask
2 hAnnotations: list[int], dict[str, int], X | None, Callable, TypeVar, Protocol, TypedDict, Literal
2 hRun mypy --strict on your Block 2 port and fix every error
1 hOpen python/pyspark/sql/ in the Spark repo and read how functions are annotated

Done when: mypy --strict reports zero errors on your own code and you can explain why a type hint does not stop a program from running. Details in Typing, mypy and coverage.

Block 4: coverage.py (4 h) ​

TimeTask
1.5 hcoverage run -m unittest, coverage report -m, coverage html, and --branch
1.5 hHow it works: coverage hooks the interpreter's tracing, records executed lines in a .coverage file and compares them with the lines that could run
1 h# pragma: no cover, then get your Block 2 port to 100% line and branch coverage
0.5 hSearch the Spark repo for a coverage runner script (python/run-tests-with-coverage, verify it exists) and read how it is used

Done when: you can explain in two minutes how coverage knows which lines ran.

Block 5: rehearse the Spark PR (6 h) ​

TimeTask
1.5 hpip install pyspark, create a local session and try a few DataFrame operations
1 hWith the build finished, run one PySpark test file with python/run-tests --testnames
1 hRead "Contributing to PySpark" and the Spark contributing guide; browse JIRA issues labeled starter
2.5 hRehearse the flow on a throwaway branch: a small change, a test, the style check, the PR title format, the JIRA ID in the test comment

Done when: one PySpark test file runs from your terminal and you can recite the PR title format.

Day-by-day schedule ​

Assuming you start on Thursday evening and the event is on Sunday:

SlotHoursBlocks
Thursday nightabout 50, then the start of 1
Fridayabout 11rest of 1, then 2 and 3
Saturdayabout 114 and 5
Sunday morningabout 1.5Verify the setup checklist, pack, leave

After the sprint ​

  • Answer review comments fast. Spark's guide asks contributors to stay responsive, and a PR with no attention is best improved and re-pinged after a few days, or made smaller.
  • Keep the habit. Reviewing other people's PRs and answering questions on the mailing list are valid contributions, per Spark's own guide.
  • Apache Airflow is a natural next Python-heavy project. Its contributing docs include a beginner quick start, static-check and testing guides, and a #new-contributors Slack channel for first-PR questions. It was not part of this sprint, and its code base is advanced Python, so treat it as a follow-up, not as study material for these 30 hours.

Unofficial community guide. Not affiliated with PyCon TW, the ASF or Apache Spark.