KNAS Green Neural Architecture Search
1 minute read ∼ Filed in : A paper noteThe paper tries to evaluate architecture without training based on the Gram Matrix of gradients of a mini-batch.
provement
Overall, the paper tries to find how the loss is related to the gradient of a mini-batch.
Finally, the paper se the mean of GM - MGM - to evaluate an architecture.
In practice, since the weight is too long, so the paper use sampling to only use a party of weights.
Algorithm
Evaluation
The result shows that MGM has a good coefficient with real performance.
This result shows that the
- KNAS is faster than search-based and gradient-based evaluation algorithms, and also has a good performance than them.
- KNAS is slower than training-free based algorithm but has better Acc on ImageNet than those 2.