Skip to content

Commit 122fa57

Browse files
llorraccllorracc
committed
CDC edited presentation
1 parent b46a917 commit 122fa57

File tree

1 file changed

+218
-87
lines changed

1 file changed

+218
-87
lines changed

‎presentation.ipynb

Lines changed: 218 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -2,102 +2,224 @@
22
"cells": [
33
{
44
"cell_type": "markdown",
5-
"id": "54ab2ec5-3d8c-4e0d-b447-db12bdf670f1",
6-
"metadata": {},
5+
"id": "18c26213-20d1-48b8-b498-3cba2b5997fd",
6+
"metadata": {
7+
"editable": true,
8+
"slideshow": {
9+
"slide_type": ""
10+
},
11+
"tags": []
12+
},
713
"source": [
8-
"# Reproducibility Standards in Economics\n",
9-
"\n",
10-
"## JHU FOSSProF Summative Event --- October 7, 2024\n",
11-
"\n",
12-
"### Alan Lujan ([email protected])\n",
13-
"\n",
14-
"### Chris Carroll ([email protected])\n",
14+
"<h1 style=\"text-align: center;\"><a href=\"https://github.com/econ-ark/FossProf\">Reproducibility Standards</h1>\n",
15+
"<h2 style=\"text-align: center;\">in Economics (and beyond)</h2>\n",
16+
"<h3 style=\"text-align: center;\">JHU FOSSProF Summative Event</h3>\n",
17+
"<h5 style=\"text-align: center;\">October 7, 2024</h5>\n",
1518
"\n",
16-
"![title](econ-ark-logo.png)"
19+
"<h4 style=\"text-align: center;\">Alan Lujan ([email protected])</h4>\n",
20+
"<h4 style=\"text-align: center;\">Chris Carroll ([email protected])</h4>\n",
21+
"<div style=\"text-align:center\">\n",
22+
" <img src=\"econ-ark-logo.png\" alt=\"econ-ark-logo\">\n",
23+
"</div>\n"
1724
]
1825
},
1926
{
2027
"cell_type": "markdown",
21-
"id": "cfcc173a-25e5-4c8d-844d-584a445a8505",
22-
"metadata": {},
28+
"id": "534e5b77-37af-44c7-8867-1cf8d8b7cdb0",
29+
"metadata": {
30+
"editable": true,
31+
"slideshow": {
32+
"slide_type": "slide"
33+
},
34+
"tags": []
35+
},
2336
"source": [
24-
"# REMARK in a Nutshell\n",
25-
"\n",
26-
"- Economics profession lags other technical fields in reproducibility practices\n",
27-
"- State of the field: `.zip` file attached to journal publication, has *some files*\n",
37+
"# [REMARK](https://github.com/econ-ark/REMARK) Motivation\n",
2838
"\n",
29-
"## Econ-ARK's REMARK project\n",
39+
"- Econ lags in reproducibility practices\n",
40+
"- State of field: for _published_ papers\n",
41+
" - `.zip` archive has *some files*\n",
42+
" - rarely works 'out of the box'\n",
43+
" - Top-5 journals have a 'data editor'\n",
44+
" - Very expensive\n",
45+
" - Excellent barrier to competition"
46+
]
47+
},
48+
{
49+
"cell_type": "markdown",
50+
"id": "4e57b738-4c15-4342-b708-86faf1ff7905",
51+
"metadata": {
52+
"editable": true,
53+
"slideshow": {
54+
"slide_type": "slide"
55+
},
56+
"tags": []
57+
},
58+
"source": [
59+
"## [Econ-ARK](https://econ-ark.org) REMARK solutions\n",
3060
"\n",
31-
"- Set of standards and tools for reproducibility of computational modeling in economics\n",
61+
"- _Standards_ for reproducibility of compute\n",
3262
" - self-contained and complete projects\n",
3363
" - executable on any modern computer\n",
64+
" "
65+
]
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"id": "7e0c600c-34d9-4e0e-be1d-05bd0c70aad9",
70+
"metadata": {
71+
"editable": true,
72+
"slideshow": {
73+
"slide_type": "subslide"
74+
},
75+
"tags": []
76+
},
77+
"source": [
3478
"- Emphasis on:\n",
35-
" - clear documentation\n",
36-
" - testing procedures\n",
37-
" - standardized metadata\n",
38-
"- Important: want project text *and* code archived and reproduced *together*"
79+
" - minimally good code\n",
80+
" - enforced algorithmically\n",
81+
" - would be huge improvement\n",
82+
" - \"ChatGPT, fix my documentation\"\n",
83+
" - explicit claims (\"Risk Aversion: [2.5,3.5]\")\n",
84+
" - standardized metadata"
3985
]
4086
},
4187
{
4288
"cell_type": "markdown",
43-
"id": "07eb06c3-1c41-447d-91e8-588c8b542d63",
44-
"metadata": {},
89+
"id": "e99d52f5-4aeb-489c-ac6c-9958c730eac4",
90+
"metadata": {
91+
"editable": true,
92+
"jp-MarkdownHeadingCollapsed": true,
93+
"slideshow": {
94+
"slide_type": "slide"
95+
},
96+
"tags": []
97+
},
98+
"source": [
99+
"# REMARK Library: [About 23+3](https://econ-ark.org/materials)\n",
100+
"\n",
101+
"- Many replicating canonical papers\n",
102+
"\n",
103+
"- Three new ones created for FossProF:\n",
104+
" - [Imai_Keane (2004) by John Green](https://github.com/JohnRGreen/ImaiKeane_replication)\n",
105+
" - [DeNardi (2004) by Ashish Kumar](https://github.com/ashishk87/DeNardi_2004_replication)\n",
106+
" - [Aiyagari (1994) by Adam Edwards](https://github.com/Adam-Edwards-JHU/Aiyagari1994QJE)\n",
107+
" \n",
108+
"- Paul Romer:\n",
109+
" - This is the future of academic publishing"
110+
]
111+
},
112+
{
113+
"cell_type": "markdown",
114+
"id": "13a07ba2-873f-4eba-965f-d749450281c4",
115+
"metadata": {
116+
"editable": true,
117+
"slideshow": {
118+
"slide_type": "slide"
119+
},
120+
"tags": []
121+
},
45122
"source": [
46-
"# Jointly Reproducible Code and Text\n",
123+
"# _Jointly_ Reproducible:\n",
124+
"## -- Code and Math and Text\n",
47125
"\n",
48-
"- Had already solved most computational aspects with extant tools:\n",
49-
" - Docker containers\n",
50-
" - version control systems\n",
51-
"- But lacked robust and portable standard for the production of the text\n",
52-
" - Want to *directly integrate* reproducible code content in reproducible text\n",
53-
"- LaTeX is standard and widespread in economics, but...\n",
126+
"- Solved: most computational aspects\n",
127+
" - [Docker containers](https://docker.com) for universality\n",
128+
" - Linux, Windows, MacOS, AWS, ...\n",
129+
" - version control (git/[GitHub](https://github.com)) = ownership\n",
130+
" - [cff references](https://citation-file-format.github.io/) make it findable/indexable\n",
131+
" - [Zenodo](https://zenodo.org): independent \"release\" archiving"
132+
]
133+
},
134+
{
135+
"cell_type": "markdown",
136+
"id": "5541a29d-73fd-44de-858e-ee8c2753d26b",
137+
"metadata": {
138+
"editable": true,
139+
"slideshow": {
140+
"slide_type": "slide"
141+
},
142+
"tags": []
143+
},
144+
"source": [
145+
"# Math and Text _with_ Code?\n",
146+
"- [LaTeX](https://www.tug.org/texlive/) is standard for text, but...\n",
54147
" - definitely *not lightweight*\n",
55-
" - only produces static documents\n",
56-
" - can't incorporate dynamically executable content"
148+
" - no dynamically executable content\n",
149+
"- Now: tenuous connection _between_ text and code\n",
150+
" - \"where is equation 18 implemented\"?\n",
151+
" - in a gestalt of interactions\n",
152+
"- Future:\n",
153+
" - [\"this math: implemented by this code\"](https://econ-ark.org/materials/bufferstocktheory)"
57154
]
58155
},
59156
{
60157
"cell_type": "markdown",
61158
"id": "ff00199e-022c-4050-a0f9-0b38ba7c68f3",
62-
"metadata": {},
159+
"metadata": {
160+
"editable": true,
161+
"slideshow": {
162+
"slide_type": "slide"
163+
},
164+
"tags": []
165+
},
63166
"source": [
64-
"# REMARK + FOSSProF --> CurveNote\n",
167+
"# FOSSProF $\\rightarrow$ [CurveNote](https://curvenote.com/)\n",
65168
"\n",
66-
"FOSSProF funding used to hire open source software contractor\n",
169+
"Funding let us hire open source contractor\n",
170+
"- To do things we could not do\n",
67171
"\n",
68-
"## CurveNote work at direction of Econ-ARK team:\n",
69-
"- Integrated MyST Markdown into REMARK project\n",
70-
"- Filled gaps in OS tools for typesetting and integrating results *in text*\n",
71-
"- Made improvements to MyST itself: increase backward compatibility w/ LaTeX\n",
72-
"- Can execute reproducible code directly in MyST documents"
172+
"## At our direction:\n",
173+
"- Integrated [MyST Markdown](https://mystmd.org/) into REMARK\n",
174+
"- Filled gaps in tools to integrate text, math, and code\n",
175+
"- Improved LaTeX $\\rightarrow$ MyST engine\n",
176+
" - Tools required for some of our REMARKs"
73177
]
74178
},
75179
{
76180
"cell_type": "markdown",
77181
"id": "baf2c769-f603-450c-9e62-c4c8c2444895",
78-
"metadata": {},
182+
"metadata": {
183+
"editable": true,
184+
"slideshow": {
185+
"slide_type": "slide"
186+
},
187+
"tags": []
188+
},
79189
"source": [
80-
"# CurveNote progress: LaTeX improvements\n",
190+
"# CurveNote: LaTeX\n",
81191
"\n",
82192
"![title](CNlatexImprovements.png)"
83193
]
84194
},
85195
{
86196
"cell_type": "markdown",
87197
"id": "9b2fa14e-1122-476c-a57c-d30587a33df2",
88-
"metadata": {},
198+
"metadata": {
199+
"editable": true,
200+
"slideshow": {
201+
"slide_type": "slide"
202+
},
203+
"tags": []
204+
},
89205
"source": [
90-
"# CurveNote progress: Integration with REMARK examples\n",
91-
"\n",
206+
"# CurveNote: Computation\n",
92207
"![title](CNexamples.png)"
93208
]
94209
},
95210
{
96211
"cell_type": "markdown",
97212
"id": "81280fcf-d738-47a6-9f69-69228898caa4",
98-
"metadata": {},
213+
"metadata": {
214+
"editable": true,
215+
"jp-MarkdownHeadingCollapsed": true,
216+
"slideshow": {
217+
"slide_type": "slide"
218+
},
219+
"tags": []
220+
},
99221
"source": [
100-
"# Demo: Journal of Open Source Economics\n",
222+
"## Journal of Open Source Economics\n",
101223
"\n",
102224
"<a href=\"https://josecon.curve.space/articles/018dc338-e64c-7c68-9b37-4f3d092f4252\" target=\"_blank\">\n",
103225
" <img src=\"compute.png\" />\n",
@@ -106,56 +228,65 @@
106228
},
107229
{
108230
"cell_type": "markdown",
109-
"id": "79e16937-1ed7-4edb-88fb-4a181573e580",
110-
"metadata": {},
231+
"id": "eee5306c-5365-4e68-a093-685dca29e4af",
232+
"metadata": {
233+
"editable": true,
234+
"slideshow": {
235+
"slide_type": "slide"
236+
},
237+
"tags": []
238+
},
111239
"source": [
112-
"# The Work Continues...\n",
113-
"\n",
114-
"### Further MyST improvements for LaTeX compatibility\n",
115-
"\n",
116-
"- Improved table \\& references formatting\n",
117-
"- Expand extensibility / external package compatibility\n",
240+
"# Not Specific to Economics!\n",
118241
"\n",
119-
"### Command-line interface tool\n",
120-
"\n",
121-
"- Used to generate REMARK templates, check for compliance with standard\n",
122-
"- Functionality for building environments and running reproducibility scripts\n",
123-
"\n",
124-
"### Making more REMARKS\n",
125-
"\n",
126-
"- JHU economics students reproduced \"classic\" papers as REMARKs\n",
127-
"- *Original* code and text of older papers should be properly archived"
242+
"- conceived for projects using [Econ-ARK](https://docs.econ-ark.org)\n",
243+
" - ... but doesn't require Econ-ARK\n",
244+
"- Not bound to economics *in any way*"
128245
]
129246
},
130247
{
131248
"cell_type": "markdown",
132-
"id": "899c9183-5dd2-47a3-af71-07abf6f8b7dd",
133-
"metadata": {},
249+
"id": "90847c77-e686-49d6-a7ea-04f0bc84435f",
250+
"metadata": {
251+
"editable": true,
252+
"slideshow": {
253+
"slide_type": "slide"
254+
},
255+
"tags": []
256+
},
134257
"source": [
135-
"# Oops: No Economics!\n",
136-
"\n",
137-
"- REMARK conceived for projects using Econ-ARK's toolkit... but doesn't use it\n",
138-
"- In fact, REMARK not bound to economics *in any way*\n",
139-
"\n",
140-
"## Rebranding: REMARK --> SCI-PASS\n",
258+
"## Rebranding: REMARK $\\rightarrow$ SCI-PASS\n",
141259
"\n",
142260
"- **S**cholarly **C**ommunication **I**nfrastructure for **P**ublishing and **A**rchiving **S**cientific **S**oftware\n",
143-
"- Captures ambition to establish universal \\& inclusive standard for reproducible scientific software\n",
144-
"- Also short for “**sci**entific **pass**port”: portability and exchangeability of research outputs\n",
145-
"\n",
146-
"## FUTURE AMBITIONS GO HERE\n",
147-
"\n",
148-
"- I don't know what you want to say\n",
149-
"- I bet you have something very clever"
261+
"- Ambition:\n",
262+
" - universal standard for repr compute\n",
263+
" - nothing now exists (we've looked)\n",
264+
"- Also short for “**sci**entific **pass**port”\n",
265+
" - portability/sharing of research outputs"
150266
]
151267
},
152268
{
153-
"cell_type": "code",
154-
"execution_count": null,
155-
"id": "4e17e52e-546a-42eb-8be0-3070fca925b7",
156-
"metadata": {},
157-
"outputs": [],
158-
"source": []
269+
"cell_type": "markdown",
270+
"id": "72738d09-79ac-4e31-b5e5-2a0c13600553",
271+
"metadata": {
272+
"editable": true,
273+
"slideshow": {
274+
"slide_type": "slide"
275+
},
276+
"tags": []
277+
},
278+
"source": [
279+
"## Long-Term Goal: ISO standard\n",
280+
"\n",
281+
"- Adoptable by journals, libraries, archives ...\n",
282+
"- Existing software standards got started this way\n",
283+
" - Somebody needed it for their own purposes\n",
284+
"- Members of our team:\n",
285+
" - SB: \"Information Science\" PhD\n",
286+
" - connection to archivists, librarians\n",
287+
" - AS: Economics PhD\n",
288+
" - Vetted ISO software standards for Oz"
289+
]
159290
}
160291
],
161292
"metadata": {
@@ -174,7 +305,7 @@
174305
"name": "python",
175306
"nbconvert_exporter": "python",
176307
"pygments_lexer": "ipython3",
177-
"version": "3.10.14"
308+
"version": "3.10.13"
178309
}
179310
},
180311
"nbformat": 4,

0 commit comments

Comments
 (0)