Hyperparameter Optimization of GPT-2 for Enhanced Text Generation

The rapid advancement of generative language models has sparked a growing interest in balancing creativity and consistency in text generation. While many of the latest models are publicly accessible, their training methods and datasets remain undisclosed. However, older models such as GPT-2 provide full documentation on their training process, making them suitable for investigating how hyperparameter configurations influence output quality. This study evaluates the effects of temperature, top-p, top-k, beam-search and greedy-search. To assess the final outputs, Distinct-N and BERTScore metrics have been used, which measure textual diversity and semantic alignment, respectively. Each parameter was systematically varied, and the resulting texts were analyzed to generate visual representations identifying the configurations that yield coherent and diverse outputs. This research contributes to a better understanding of how hyperparameter tuning can enhance the adaptability and output of the GPT-2 model.