functor by default

[CarloLucibello]

Makes everything a functor by default, avoiding the need to sprinkle

@functor T

everywhere in Flux's layers and similar use cases.
The types already decorated with @functor T or @functor T (a, b) won't be affected by the change.

The amount of breakage and unintended consequence this PR could produce is something I cannot estimate at the moment.

Fix #49

TODO:

• docs
• test with Flux.jl

[mcabbott]

The fact that there's still a method for AbstractArray{<:Number} means that it doesn't recurse into the reshape here, which is good I think:

julia> pr(x) = (@show typeof(x); x);

julia> fmap(pr, rand(3)');
typeof(x) = Vector{Float64}

julia> fmap(pr, reshape(rand(Int8, 4)',2,2))
typeof(x) = Base.ReshapedArray{Int8, 2, Adjoint{Int8, Vector{Int8}}, Tuple{}}
2×2 reshape(adjoint(::Vector{Int8}), 2, 2) with eltype Int8:
   53  -63
 -125  -58

The default functor doesn't seem able to reconstruct closures like:

julia> D = let W = rand(2,2), b = zeros(2)
             x -> tanh.(W*x .+ b)
           end
#11 (generic function with 1 method)

julia> fmap(pr, D)
typeof(x) = Matrix{Float64}
typeof(x) = Vector{Float64}
ERROR: MethodError: no method matching var"#11#12"(::Matrix{Float64}, ::Vector{Float64})
Stacktrace:
 [1] (::Functors.var"#3#6"{UnionAll})(y::NamedTuple{(:W, :b), Tuple{Matrix{Float64}, Vector{Float64}}})
   @ Functors ~/.julia/packages/Functors/1AaAn/src/functor.jl:8
 [2] (::Functors.DefaultWalk)(::Function, ::Function)
   @ Functors ~/.julia/packages/Functors/1AaAn/src/walks.jl:56
...

julia> fieldnames(var"#11#12")
(:W, :b)

julia> methods(var"#11#12")
# 0 methods for type constructor

In global scope, example from https://fluxml.ai/Flux.jl/stable/models/basics/#Building-Simple-Models just does nothing instead, unsurprisingly:

julia> W = rand(2, 5);
julia> b = rand(2);
julia> predict(x) = W*x .+ b;

julia> fmap(pr, predict)
typeof(x) = typeof(predict)
predict (generic function with 1 method)

julia> fieldnames(typeof(predict))
()

[mcabbott]

I think this will be slow. In FluxML/Flux.jl#1932 it needed a generated function to be as quick as before:

https://github.com/FluxML/Flux.jl/pull/1932/files#diff-34ea0dd342eeb0864015f5ab93bd9db67f6f46ec46d4bf8dbebe9b8c1a433b04R129-R141

[CarloLucibello]

let's keep this in mind for a future optimization-oriented PR

[mcabbott]

I think Kyle said he had a branch doing something like this, using ConstructionBase. That appears to be able to reconstruct closures which is neat:

julia> adder = let y = ones(1)
                 x -> x .+ y
               end
#38 (generic function with 1 method)

julia> adder.y
1-element Vector{Float64}:
 1.0

julia> adder(2)
1-element Vector{Float64}:
 3.0

julia> using ConstructionBase

julia> newadder = constructorof(typeof(adder))([4 5 6])
#38 (generic function with 1 method)

julia> newadder(2)
1×3 Matrix{Int64}:
 6  7  8

It is happy to re-build things like reshape(rand(Int8, 4)',2,2), not sure how that will interact with fmap(f, x, dx) for gradients etc.

[ToucheSir]

I learned recently that it's possible to de- and reconstruct closure types. See JuliaGPU/Adapt.jl#58 for an implementation of this. Even if we can't functor all user-defined types by default, maybe this is something we could in a backwards-compatible way? I could see it as a pilot project of sorts too.