Request for Papers
Research Topics in Compute History & Policy
I recently wrote a 12-page history of compute and numerical weather prediction from the 1950s to 1970s.1 Following some encouragement from
, I’m writing a list of related compute research topics I wish I had time to explore.
Some of these questions were ones I stumbled upon during the course of writing the compute & numerical weather prediction paper. Others are ones I’ve stored up during my time working at the Department of Energy. As a result, they are obviously biased based on my policy-oriented, often DOE-specific perspective. If you also have many ideas and not enough time, whether in compute or other unexplored topics, you should publish your own list! 2
If you’re a young person looking to break into AI policy—or just looking for something worthwhile to pursue this summer—I can think of few better ways than writing publicly about an interesting question you’re exploring. Finding good questions can often be harder than finding good answers. I hope this list offers a useful starting point.
Table of Contents:
How did universities, NSF, and Congress come together to launch the 1985 NSF Supercomputer Program?
How has DOE’s supercomputer procurement shaped the U.S. compute industrial base since the 1990s?
Did the 1984 Competitiveness in Contracting Act (CICA) undermine DARPA’s effectiveness?
How did U.S. supercomputer export controls evolve during the Cold War?
How has Nvidia’s technology stack benefited from public R&D partnerships?
What are the economic and scientific returns to public supercomputing?
How did universities, NSF, and Congress come together to launch the 1985 NSF Supercomputer Program?
This is not the first time U.S. university researchers have been compute limited compared to their peers. In the 1980s, U.S. universities faced a “supercomputer famine” compared to nuclear weapon labs and European peers. A push from computer science researchers led the National Science Foundation (NSF) to create a supercomputer program in 1985, funding the construction of supercomputers at five universities across the country. Notably, this program is what allowed Marc Andreassen to start Mosaic while at UIUC, one of the 5 universities granted a supercomputer due to the advocacy of one of its faculty, Larry Smarr. The legacy of this program still lives on today, including through the NAIRR.
Who were the key players during this time, in universities, NSF, Congress, and elsewhere, and how did they convince Congress to appropriate funding? What medium-term R&D benefits emerged from the development of university-based compute capabilities? What can we learn from the successful university-based advocacy of the 1980s and what has changed today?
Interview with Larry Smarr, UIUC professor and founder of National Center for Supercomputing Applications
1985 Supercomputer Congressional Hearing (where the cover image for the post is from)
1991 High Performance Supercomputing Act, introduced by Al Gore
How has DOE’s supercomputer procurement shaped the U.S. compute industrial base since the 1990s?
The Department of Energy national labs have long been a leader in building high-performance computing systems, an effort which particularly took off after the Comprehensive Nuclear-Test-Ban Treaty in 1996. Tremendous amounts of scientific data on high-energy physics were collected from nuclear weapon testing. If you can no longer collect that data in-situ, you must move to in-silico. And thus, DOE embarked on the task of building the world’s largest supercomputers.
In DOE’s procurement of supercomputers, which are based across the national lab complex, DOE’s procurement system has been described to me as remarkably similar to DoD’s. Specifically, DOE doesn’t just procure the best system; it deliberately spreads contracts across vendors to support a broader compute industrial base.3 As I argue in my compute & numerical weather prediction paper, public applications of compute have long served as demand-pull and customer-of-first-resort mechanisms to support a domestic compute industry.
What commercial technologies did DOE help pull forward through its supercomputer procurement program? How does DOE national lab procurement help support a more competitive compute industrial base, including the AI accelerator startup ecosystem? How does thinking about public capital and demand-pull support for highly capital intensive industries shape how we think about antitrust policy toolkit? How should the U.S.’s role in supporting compute innovation evolve in light of the recent increase in private capital?
(for inspiration) Japanese Industrial Policy: the Postwar Record and Supercomputers
Did the 1984 Competitiveness in Contracting Act (CICA) undermine DARPA’s effectiveness?
DARPA played a pivotal role in developing early parallel and intelligent computing systems. There is some debate around whether DARPA is still as effective as it once was. I have seen some hints that one undertheorized cause of potential bureaucratic bloat at DARPA is the 1984 Competitiveness in Contracting Act (CICA), which ostensibly promoted “competitiveness,” but largely increased paperwork to prove contracts were competitive, often at the expense of focusing on outcomes. DARPA could serve as a case study on the impacts of this substantial change to federal contracting, an exceedingly mundane policy topic but the “dark matter” behind much of policymaking’s success or failure. Such a piece would particularly be of interest given the renewed interest in state capacity and abundance-adjacent discourse.
What were the concrete changes in contracting mechanisms before and after CICA at DARPA? Are there first-hand or second-hand evidence from the time that such changes negatively impacted DARPA’s ability to execute quickly (see Freaktakes link for one example)?
How did U.S. supercomputer export controls evolve during the Cold War?
One major update for me in writing my compute & numerical weather prediction paper is realizing that this is not the first time the U.S. has engaged in compute export control. The U.S. literally had FLOP-based compute thresholds, on a country-by-country basis, throughout the Cold War. That said, I haven’t found much pre-1990s writing available online that covers this history in depth. But you can see the effects of it, from de Gaulle’s Plan Calcul to the collapse of UK computing industry. I suspect it might be paywalled behind some JSTOR article or a long history on COCOM, but someone should liberate this data!
What drove the changes behind shifting US compute export control regimes? What was the shape of US political economy and export controls back then and how did compute export controls relate to nuclear or other export technologies? What were the broader market effects of waning and waxing US export control on other county’s domestic compute industry development? What lessons can be learned from this period of US export control history?
1999 Report of the Select Committee on U.S. National Security and Military/Commercial Concerns with the People's Republic of China, Chapter 3, High Performance Computing
Which countries are funding autonomous experimentation platforms, through what mechanisms, and at what scale?
If AI models will commoditize hypothesis generation/inference, then the rate limiting bottleneck becomes testing and verification. In my view, autonomous experimentation platforms, or self-driving labs, are therefore a critical technology platform to realize the acceleratory benefits of AI for scientific discovery. Despite this, no one has done a comprehensive overview comparing different countries funding for self-driving labs. 4
How much funding has each country allocated towards autonomous experimentation? How much of the funding is application-driven e.g. catalyst discovery that happens to leverage autonomous experimentation, platform-focused e.g. autonomous experimentation for catalyst discovery, or other e.g. NSF early career grant, national lab directed R&D?
My other substack on self-driving labs and policy/funding changes I helped push around self-driving labs when I was at DOE
UToronto Acceleration Consortium $200M funding for autonomous experimentation
How has Nvidia’s technology stack benefited from public R&D partnerships?
If you’re interested in the benefits of public R&D spending, Nvidia’s history is worth digging into. One could argue their two main moats today, CUDA and NVLink, were both developed through public R&D spending. Concretely, Nvidia has a long history of close partnership with DOE national labs. The DOE supercomputers help derisk new compute technologies at scale, including being some of the first adopters of liquid cooling. Someone should do the work of telling that story.
2004 BrookGPU paper, predecessor to CUDA, was funded by DARPA and National Nuclear Security Administration [screenshot]
2017 DOE budget report on exascale computing research with Nvidia that led to their NVLink [screenshot]
What are the economic and scientific returns to public supercomputing?
For my economics friends.
Having more compute for scientific research seems generally good. But I don’t think there has been enough quantification of these benefits and in particular, I know DOE allocations have been understudied relative to their impact.
How have allocations on DOE supercomputers changed over time in terms of subject area, use of AI, project team size, etc? How does receiving an allocation change a researcher trajectory? Running a workload on a supercomputer is non-trivial - how well do DOE programs derisk the challenges for new researchers working on supercomputers? How should public compute allocation strategies change in light of AI & scaling laws?
Underleveraged DOE data resources include: DOE INCITE Awards, DOE ALCC Awards, DOE user facility statistics (filter by supercomputer programs). Topic-specific ones include HPC for energy and manufacturing.
Innovation under Resource Constraints: supercomputing in scientific research
Digital Dark Matter and the Economic Contribution of Apache software
If you are interested in actually writing about any of these topics:
consider trying out learning ledger, a new project I’m hosting that encourages public writing as learning
feel free to reach out! If you do reach out, please explicitly demonstrate that you have put some more time and effort into thinking about and researching these questions than just reading this post, and provide any specific questions you are looking for clarity on.
If you make progress on writing something and would like to publish, feel free to send me a draft and I’m happy to potentially publish/suggest appropriate outlets
This was actually quite a fun project. I think in another life, I would have loved to be a history of science grad student digging through archives. Discovering something new about the past that directly, almost obviously, relates to questions of today, is a special feeling!
I may be missing existing sources. It would be great to surface semi-contemporary work that already addresses these topics. Please comment or reply if you know of any.
For instance, note DOE’s top 3 supercomputers (which also happen to be top 3 on the Top500 HPC list) are AMD, AMD, and Intel respectively.
I keep a mental list in my head but haven’t had the time to flesh it out. In particular, I have no visibility on what China is doing in this area. Canada is a clear leader, U.S. only recently in the last 9 months dedicated programmatic funding towards self-driving labs (which is now on pause along with broader U.S. science funding…), the U.K. has a few good researchers as does E.U. I’ve mostly pieced this together by digging through funding acknowledgements, but a more systematic analysis remains wide open.