Faster Convergence for Transformer Fine-tuning with Line Search Methods.
Philip KennewegLeonardo GalliTristan KennewegBarbara HammerPublished in: CoRR (2024)
Keyphrases
- search methods
- fine tuning
- faster convergence
- search algorithm
- step size
- global optimum
- viable alternative
- convergence speed
- search strategies
- metaheuristic
- exhaustive search
- fine tune
- unconstrained optimization
- systematic search
- pso algorithm
- fuzzy logic
- global optimization
- fine tuned
- fault diagnosis
- beam search
- search space
- convergence rate
- particle swarm optimization
- search procedures
- heuristic function
- optimization algorithm
- nearest neighbor search
- path planning
- differential evolution
- heuristic functions
- hessian matrix
- objective function
- genetic algorithm