This work describes the main idea of Probabilistic Programming, which is a new approach to statistical modelling, that aims to simplify working with inference problems. It discusses the basics of statistics, which are needed for working with Probabilistic Programming, including the Bayesian interpretation of probability. After this it introduces the forward and the backward direction of Probabilistic Programming, i.e. the simulation of data, and the inference for the parameters of a model. It also contains a short introduction to Machine Learning, a domain which uses Probabilistic Programming more than others.

Additionally it lists a selection of 16 Probabilistic Programming Systems and explains briefly their most important facts and features. It also includes an introduction to the following three Probabilistic Programming Systems: Infer.NET, BUGS, and Church. The introduction includes amongst the main characteristic features a simple example of how to create a model, and do inference on it. At the end of this work the Probabilistic Programming Languages are analyzed, and their programming concepts and their programming paradigms are extracted. After that the taxonomy of traditional programming paradigms is extended by the main concept of Probabilistic Programming, the random choice, to create the new Probabilistic Programming paradigm.