Skip to content

docs: the bitter lesson #439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/website/src/index.html.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@
<a href="https://twitter.com/intent/tweet?url=https://hacker-laws.com/#{{ section.id}}?hashtags=example" title="Share on Twitter" target="_blank"><i class="bi bi-twitter"></i></a>
<a href="https://www.facebook.com/sharer/sharer.php?u=https://hacker-laws.com/&num;{{ section.id }}" title="Share on Facebook" target="_blank"><i class="bi bi-facebook"></i></a>
</div>
<div class="back-to-top"><a href="#top">↑ Back to Top</a></div>
</section>
{% endfor %}

Expand Down
3 changes: 3 additions & 0 deletions .github/website/src/script.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,7 @@ $(document).ready(function() {
// Append to the heading
$heading.append($anchor);
});

// Bootstrap requires that blockquote elements have the 'blockquote' class.
$('blockquote').addClass('blockquote').addClass('.quote');
});
7 changes: 7 additions & 0 deletions .github/website/src/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,24 @@ body {
.container {
max-width: 800px;
}

header {
text-align: center;
margin-bottom: 2rem;
padding: 2rem 0;
border-bottom: 1px solid #e5e5e5;
}

h1, h2, h3, h4, h5, h6 {
font-family: 'Libre Baskerville', serif;
/* Avoid scrolling under the sticky header. */
scroll-margin-top: 80px;
}

blockquote {
font-style: italic;
}

.law-section {
margin-bottom: 2rem;
padding: 1.5rem;
Expand Down
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
---

- 📖 Check out my new book [Effective Shell](https://effective-shell) on [Amazon](https://amzn.to/4ho0F91)
- 🌍 Check out the website [hacker-laws.com](https://hacker-laws.com)
- 🌍 Try [hacker-laws.com](https://hacker-laws.com)
- 🧠 Experiment with my new project [Terminal AI](https://github.com/dwmkerr/terminal-ai)
- ☕️ Like this project? Consider [buying me a coffee with a one-off donation](https://github.com/sponsors/dwmkerr?frequency=one-time)
- 🧠 Check out my new project [Terminal AI](https://github.com/dwmkerr/terminal-ai)
- 🎧 Try the podcast [The Changelog - Laws for Hackers to Live By](https://changelog.com/podcast/403)
- 🎧 Listen to the podcast [The Changelog - Laws for Hackers to Live By](https://changelog.com/podcast/403)
- 📖 Download the [PDF eBook](https://github.com/dwmkerr/hacker-laws/releases/latest/download/hacker-laws.pd)

---
Expand Down Expand Up @@ -47,6 +47,7 @@
- [Premature Optimization Effect](#premature-optimization-effect)
- [Putt's Law](#putts-law)
- [Reed's Law](#reeds-law)
- [The Bitter Lesson](#the-bitter-lesson)
- [The Ringelmann Effect](#the-ringelmann-effect)
- [The Law of Conservation of Complexity (Tesler's Law)](#the-law-of-conservation-of-complexity-teslers-law)
- [The Law of Demeter](#the-law-of-demeter)
Expand Down Expand Up @@ -598,6 +599,18 @@ See also:
- [Metcalfe's Law](#metcalfes-law)
- [Dunbar's Number](#dunbars-number)

### The Bitter Lesson

[The Bitter Lesson by Richard S. Sutton](http://www.incompleteideas.net/IncIdeas/BitterLesson.html)

> The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
>
> Richard S. Sutton (2019)

The "Bitter Lesson", stated by [Rich S. Sutton](https://en.wikipedia.org/wiki/Richard_S._Sutton), says that scale (in terms of both data and computational power) has driven the most significant advancements in AI research, rather than the intricacies of the research methods themselves.

He goes on to suggest that this indicates we should stop trying to build simplified (or even complex) models of the mind as history has shown that these have always in the long term been failures compared to (as an example) scaling the capacity of neural networks and applying existing methods such as convolution.

### The Ringelmann Effect

[The Ringelmann effect on Wikipedia](https://en.wikipedia.org/wiki/Ringelmann_effect)
Expand Down
Loading