The purpose of this book is twofold: first, it sets out to equip the reader with a sound understanding of the foundations of probability theory and stochastic processes, offering step-by-step guidance from basic probability theory to advanced topics, such as stochastic differential equations, which typically are presented in textbooks that require a very strong mathematical background. Second, while leading the reader on this journey, it aims to impart the knowledge needed in order to develop algorithms that simulate realistic physical systems. Connections with several fields of pure and applied physics, from quantum mechanics to econophysics, are provided. Furthermore, the inclusion of fully solved exercises will enable the reader to learn quickly and to explore topics not covered in the main text. The book will appeal especially to graduate students wishing to learn how to simulate physical systems and to deepen their knowledge of the mathematical framework, which has very deep connections with modern quantum field theory.