Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve GECO #3

Open
dalessioluca opened this issue Apr 29, 2020 · 4 comments
Open

Improve GECO #3

dalessioluca opened this issue Apr 29, 2020 · 4 comments
Assignees

Comments

@dalessioluca
Copy link
Collaborator

what is the range of allowed value fo the GECO hyper-parameters (0; + infinity) ?
change of hyper-parameters should be proportional to the distance to the target

@dalessioluca
Copy link
Collaborator Author

87 "factor_nll_range" : [0.0, 1.0, 40.0], this should be allowed to become negative to prevent empty solution
88 "factor_sparsity_range" : [0.0, 1.0, 40.0], this should be stricntly positive

@dalessioluca
Copy link
Collaborator Author

dalessioluca commented May 16, 2020

I realized that sparsity should be controlled by fg_pixel_fraction (which should be removed from regularization dictionary).

Moreover the balance between reconstruction and KL should be that
a = b
Think about the edge cases when KL ->0
Therefore a,b should not go to zero.

maybe:
a = max(1, NLL_av /( NLL_av + KL_av) )
similarly for b

@dalessioluca dalessioluca self-assigned this May 16, 2020
@dalessioluca
Copy link
Collaborator Author

dalessioluca commented Jun 13, 2020

In PR #20:

  1. sparsity is adjusted based on fg_fraction
  2. balance between KL and NLL is adjusted based on the NLL target

KL_LOGIT is normalized w.r.t to running average. The only question is wether the other KL should also be normalized in the same way......

@dalessioluca
Copy link
Collaborator Author

Interesting observation:
NLL is intensive and of order 1.
Sparsity is also intensive and of order 1.
If I switch off a foreground pixel I am going to pay a price in NLL but gain in SPARSITY....
It seems that these procedure will reach an equilibrium in which:
alpha = beta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant