Lecture VIII: Your Research Project

AI-Assisted Programming for PhD Researchers

Dr. Tobias Vlćek

Helmut Schmidt University, August 2026

Research Code Is Different

The Code Is the Claim

In research, code correctness is result validity.

• A bug here is not a crash but a wrong Table 1
• Remember Lab 3: one silent fillna shifted every summary statistic
• Nobody sees a stack trace; they see a number that looks fine and is not
• The code and the result stand or fall together

Reproducibility Checklist

Four things make a result someone else can rerun:

• Pinned environment: uv lock, renv, or Project.toml
• Fixed seeds: every random draw is deterministic
• Data versioned or archived: findable and persistent (Wilkinson et al. 2016)
• One command from raw data to every figure

Notebooks: Handle with Care

• Hidden state and out-of-order execution quietly wreck reproducibility
• Of ~1.16M public notebooks Pimentel et al. mined, only ~4% reproduced their own saved output (Pimentel et al. 2019)
• 36% ran cells out of order (a contributing cause, not the failure rate itself) (Pimentel et al. 2019)
• Fine for exploration; the record should be a pipeline, like the one you built

Agents and Reproducibility

Agents are good at exactly this cleanup work:

• “Make this run end-to-end from scratch in a clean environment”
• “Pin my dependencies and generate a lock file”
• “Find the hidden state in this notebook and flag out-of-order cells”

Zed’s Built-in REPL

• Zed can run code inline, cell by cell, in an ordinary .py file, no separate notebook file at all
• Backed by Jupyter kernels. Same execution model as Jupyter, but your code stays an ordinary script: no saved outputs, nothing a .ipynb can hide
• The kernel needs to be installed in your project’s environment first: uv add --dev ipykernel, then repl: refresh kernelspecs
• Worth exploring after the course for fast exploratory analysis

Remote Development over SSH

• Zed can edit and run code directly on a remote machine (a cluster, a lab workstation) over SSH
• The UI stays local and responsive; language servers, terminals, and tasks all run on the remote
• Connect via Remote Projects (ctrl-cmd-shift-o / alt-ctrl-shift-o) or zed ssh://user@host/path
• Relevant if your research runs on HPC, worth exploring after the course

AI Beyond Code

Literature Research with Agents

The deep-research pattern: search → read → synthesize with citations.

• Point an agent at a question; it fans out, reads, and summarizes with sources
• Genuinely useful for mapping an unfamiliar field fast
• A strong starting map, not a finished bibliography

The Catch: Invented Citations

Models produce references that are confident, well-formatted, and fake:

• 55% of GPT-3.5’s citations were fabricated; GPT-4 still 18% (Walters and Wilder 2023)
• Even the real ones carried errors: 43% (GPT-3.5) / 24% (GPT-4) (Walters and Wilder 2023)
• Newer is better, not safe: GPT-4o still ~20% fabricated, worst on niche topics (Linardon et al. 2025)

Spot the Fake

Same format, same confidence. One exists, one does not:

Fabricated: Smith, J. (2019). "Deep learning for
  epidemic forecasting." Nature Methods, 16(4), 331–339.

Real:       Pimentel et al. (2019). "A large-scale study
  about quality and reproducibility of Jupyter notebooks."
  MSR 2019, 507–517.

You cannot tell from the text, so every citation gets verified.

The Verification Discipline

For every AI-suggested source, three checks:

• DOI resolves: the link actually goes somewhere
• Paper exists: real venue, real authors, real year
• It says what was claimed: read the abstract, minimum

AI does not remove the work; it moves it: from finding to verifying (Lee et al. 2025). This afternoon’s Lab 5 rule, now for your thesis.

Drafting Text with Skills

A clunky sentence through conciseness + academic-grammar:

Before: It should be noted that, in order to obtain the
  results, we made use of a large number of samples.

After:  We used a large sample.

The skills suggest; you decide what stays.

Cross-Check Everything

AI drafts are drafts: verify claims sentence by sentence, numbers against sources, citations against papers.

The deeper risk is producing more while understanding less: “illusions of understanding” (Messeri and Crockett 2024). You sign it, you own it.

Institutions Are Responding

• AI agents in research are now a policy topic: journals, funders, and institutions are writing the rules (“AI Scientists Are Changing Research — Institutions, Funders and Publishers Must Respond” 2026)
• Provider watermarks (Lecture VI) hand journals a detection tool, so expect checks to become routine
• The disclosure and verification habits from Lecture VI already keep you ahead: the requirements are only catching up

Your Capstone

The Assignment

Tomorrow you work on your project. This afternoon you plan.

• Take a real problem from your own research and point an agent at it
• Tomorrow afternoon, everyone shows what they built and what went wrong
• No slide deck needed. The working thing and an honest account of it are enough

What Makes a Good Capstone

• A real problem from your own research
• The core fits in ~2–3 hours; anything bigger goes in stretch goals
• Something checkable comes out: a figure, a table, a test suite that passes
• Runs the whole workflow: spec → plan → implement → verify

No Project? Option A

A BibTeX/DOI tool built from the Lab 5 pieces:

• Parse a messy .bib file or a list of DOIs
• Fetch metadata via the reference-lookup skill
• Drop the duplicates and write out a clean reading list
• Full brief on the capstone page

No Project? Option B

A small simulation with a parameter sweep:

• Pick a model: a queue, an epidemic, whatever fits your field
• Sweep two parameters over a grid
• Plot the results and put the numbers in a table
• Full brief on the capstone page

Scope It Now

• Open the capstone page and write your spec with the interview pattern
• By 16:30: spec + plan reviewed by a fresh agent session

The Capstone Project

Continue Your Journey

Next Up

• You now know why research code carries a higher bar, and how to keep AI honest in it
• Tomorrow, 09:00: we build.
• The Capstone Project: scope yours this afternoon
• Lecture IX: Judgment and Showing Your Work, tomorrow morning
• Course literature and references

“AI Scientists Are Changing Research — Institutions, Funders and Publishers Must Respond.” 2026. Nature 651 (8107): 853–54. https://doi.org/10.1038/d41586-026-00934-w.

Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, et al. 2025. “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.” Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama Japan), April, 1–22. https://doi.org/10.1145/3706598.3713778.

Linardon, Jake, Hannah K Jarman, Zoe McClure, Cleo Anderson, Claudia Liu, and Mariel Messer. 2025. “Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental Study.” JMIR Mental Health 12 (November): e80371. https://doi.org/10.2196/80371.

Messeri, Lisa, and M. J. Crockett. 2024. “Artificial Intelligence and Illusions of Understanding in Scientific Research.” Nature 627 (8002): 49–58. https://doi.org/10.1038/s41586-024-07146-0.

Pimentel, Joao Felipe, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. “A Large-Scale Study About Quality and Reproducibility of Jupyter Notebooks.” 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) (Montreal, QC, Canada), May, 507–17. https://doi.org/10.1109/MSR.2019.00077.

Walters, William H., and Esther Isabelle Wilder. 2023. “Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT.” Scientific Reports 13 (1): 14045. https://doi.org/10.1038/s41598-023-41032-5.

Wilkinson, Mark D., Michel Dumontier, IJsbrand Jan Aalbersberg, et al. 2016. “The FAIR Guiding Principles for Scientific Data Management and Stewardship.” Scientific Data 3 (1): 160018. https://doi.org/10.1038/sdata.2016.18.