Exploring the use of ABn testing and statistical hypothesis testing as tools for making efficient and data based business or designed decisions

Author : Herambha Deepak Joshi, Dr. Shridhar Mhalsekar

Abstract : In today’s data driven environment, decision making plays a very crucial role in deciding or predicting the future aspect of any entity or commodity. A/B testing and its extension, A/B/n testing have become powerful techniques to evaluate multiple alternatives simultaneously. However, incorrect selection of statistical tests, improper handling of multiple comparisons, and confusion between different data types often lead to misleading conclusions. A/B/n testing plays a crucial role in domains such as digital marketing, user experience optimization, e-commerce, recommendation systems, and product development. Decisions based on such experiments directly affect revenue, user engagement, and operational efficiency. Therefore, it is essential that these experiments are analyzed using statistically sound and reproducible methods A/B/n testing integrated with classical statistical hypothesis testing to create a structured, automated, and statistically rigorous decision-making framework. This paper and its novel system support both conversion-based data (binary outcomes such as click or no-click, purchase or no-purchase) and continuous data (numerical outcomes such as revenue, time, or performance metrics). Further based on the data type, the framework automatically selects appropriate statistical methods such as Chi-square tests, Z-tests, One-way ANOVA, and Welch’s t-tests etc.

Keywords : A/B testing, Chi-square tests, Hypothesis testing, One way ANOVA, Statistical significance, Welch’s t-test, Z test.

Conference Name : National Conference on Mechanical & Production Engineering (NCMPE-26)

Conference Place : Goa, India

Conference Date : 12th Apr 2026