Feature Request: Add comptime Functionality Similar to ZigLang to Dart

[dxvid-pts]

I would like to request the addition of comptime functionality to Dart, similar to what is available in ZigLang (https://ziglang.org). This feature would allow certain computations to be executed at compile time, improving performance and flexibility for developers.

Motivation

As demonstrated in a recent benchmark, constant values (const) are approximately 13-16% faster than non-constant values. Flutter Pull Request #148261. Introducing comptime would enable more operations to benefit from the performance advantages of being computed at compile time, thus enhancing the overall efficiency of Dart applications.

Example Use Cases

Fetching Terms of Service Markdown via HTTP

One practical use case for comptime is fetching static resources, such as a Terms of Service markdown file, from a remote server at compile time. This reduces runtime overhead and ensures that the resource is always available without incurring I/O costs during execution.

import 'dart:io';
import 'dart:convert';

const String termsOfService = comptime fetchTermsOfService();

@comptime
String fetchTermsOfService() {
  final url = 'https://example.com/assets/terms_of_service.md';
  final response = HttpClient()
      .getUrl(Uri.parse(url))
      .then((request) => request.close())
      .then((response) => response.transform(utf8.decoder).join());
  return response;
}

void main() {
  print(termsOfService);
}

In this example, the fetchTermsOfService function is marked with @comptime, indicating that it should be executed at compile time. The contents of the terms_of_service.md file are fetched via HTTP and assigned to the termsOfService constant, ensuring that this data is embedded in the compiled application.

Computing Fibonacci Numbers

Another classic use case for comptime is computing constant values that can be determined at compile time, such as Fibonacci numbers. This ensures that the computation is done once, at compile time, rather than repeatedly at runtime.

const int fib10 = comptime fibonacci(10);

@comptime
int fibonacci(int n) {
  if (n <= 1) return n;
  return fibonacci(n - 1) + fibonacci(n - 2);
}

void main() {
  print(fib10); // Outputs: 55
}

In this example, the fibonacci function calculates the 10th Fibonacci number at compile time, and the result is stored in the fib10 constant.

Benefits

• Performance Improvement: By allowing more computations to be performed at compile time, the runtime performance of Dart applications can be improved. This is particularly beneficial for applications with a high demand for efficiency, such as those built with Flutter.

• Compile-time Guarantees: Ensuring certain operations are completed at compile time provides additional safety guarantees. For instance, in the examples provided, it ensures that the Terms of Service markdown is always available and that Fibonacci numbers are precomputed.

• Reduced Runtime Overhead: Performing operations like HTTP requests and computationally intensive tasks at compile time reduces the load on the application at runtime, leading to faster startup times and more responsive applications.

Adding comptime functionality to Dart would be a significant enhancement in my opinion, providing developers with powerful tools to optimize their code and ensure greater performance and reliability. I believe this feature aligns with Dart's goals of providing a high-performance, productive language for modern applications.

[jakemac53]

We did experiment with what we called "enhanced const", which would expose all the synchronous parts of the language to the const evaluator (basically giving you what you are asking for here but without needing the explicit compliment keyword/annotation, and no async support).

It ends up being quite complicated to support well because of the invalidation semantics that are implied. Any function invoked at compile time invalidates anything that depends on it whenever the body of that function changes, and the same goes for any functions used by that function, etc.

Adding the comptime annotation would assist with that in some ways - making it explicit what is allowed to be used at compile time, and thus which things have the worse invalidation/compilation time behavior. We also considered similar approaches, but determined it would inevitably lead to requests to any sufficiently used package to add @comptime to its functions, and you end up in the same situation.

[scheglov]

It ends up being quite complicated to support well because of the invalidation semantics that are implied. Any function invoked at compile time invalidates anything that depends on it whenever the body of that function changes, and the same goes for any functions used by that function, etc.

To be fair, the same happens for macros too. A macro executes code from its dependencies, so depends not just on API, but on full code. So, the kernel is recompiled on any code change. From this, API of any library that applies a macro depends on full code of the macro and any macro dependencies. So, the analyzer rebuilds element models for libraries with macro applications even if some changes are in function bodies.

Speaking of macros, and the original request of this issue. Macros are already async, and while I think we don't want you to use IO during macros, technically I think it is possible right now :-)

[jakemac53]

The difference with macros is the boundary is more well defined and we expect them to be written less often. If any constant expression could call functions, that would likely get used a ton, and so we would have these invalidation semantics much more often.

[jakemac53]

Speaking of macros, and the original request of this issue. Macros are already async, and while I think we don't want you to use IO during macros, technically I think it is possible right now :-)

It is currently possible I think yes, but it is specified that it is not allowed :). We also don't have the replacement (Resource) API implemented, and it isn't the end of the world if people use dart:io right now, if they are just reading files.

[lrhn]

As mentioned, the difference between macros and comp time is that macros are specified as running as a separate program.
A comp time expression needs to be prevented from using any global non-constant/comptime state.

If a comptime expression, fx, increments a global variable, then it's not valid as a constant. compile-time operations must not have side effects, because those side effects may be important to keeping the result valid.
If a comptime expresssion reads a global variable, then that variable must have a constant value.

Generally, a comptime operation can only invoke other comptime operations, and currently there are none.
We'd have to annotate the platform APIs with operations that are comptime compatible, but in an interface based language, that's not a type-based thing. You can't do for (var i in something) unless all subtypes of the type of something have comptime declared get iterator which return values that are mutable (iterators have mutable state) that are still guaranteed to be comptime compatible, meaning it must not touch global state.

Dart is not designed to make global side effects visible in an API.
Imagine if some class's toString updated a global variable, so you couldn't be allowed to call Object.toString in general. Well, it happens that List.toString does exactly that, it uses a global stack for detecting cycles, also shared with other iterables. Might even be the same one used by jsonEncode.

That code could probably be rewritten to not (re)use a global variable, but ... The alternative would be during the stack in a zone, which is still mutable state external to the function, and blue it's updating Zone._current instead.

Which brings us to async, which is just one big heap of global state. That too could probably be rewritten to be more self-contained, so async computations can be made to run to completion at comptime and not leave lasting changes, but it's not something the API was designed for.