Home » Uncategorized

Free Book: Probability and Statistics Cookbook

The format is very similar to a BIG cheat sheet. This cookbook integrates a variety of topics in probability theory and statistics. It is based on literature and in-class material from courses of the statistics department at the University of California in Berkeley but also influenced by other sources . 

Author: Matthias Vallentin

Free Book: Probability and Statistics Cookbook

Contents

1 Distribution Overview 3

  • 1.1 Discrete Distributions . . . . . . . . . . 3
  • 1.2 Continuous Distributions . . . . . . . . 4

2 Probability Theory 6

3 Random Variables 6

  • 3.1 Transformations . . . . . . . . . . . . . 7

4 Expectation 7

5 Variance 7

6 Inequalities 8

7 Distribution Relationships 8

8 Probability and Moment Generating Functions 9

9 Multivariate Distributions 9

  • 9.1 Standard Bivariate Normal . . . . . . . 9
  • 9.2 Bivariate Normal . . . . . . . . . . . . . 9
  • 9.3 Multivariate Normal . . . . . . . . . . . 9

10 Convergence 9

  • 10.1 Law of Large Numbers (LLN) . . . . . . 10
  • 10.2 Central Limit Theorem (CLT) . . . . . 10

11 Statistical Inference 10

  • 11.1 Point Estimation . . . . . . . . . . . . . 10
  • 11.2 Normal-Based Confidence Interval . . . 11
  • 11.3 Empirical distribution . . . . . . . . . . 11
  • 11.4 Statistical Functionals . . . . . . . . . . 11

12 Parametric Inference 11

  • 12.1 Method of Moments . . . . . . . . . . . 11
  • 12.2 Maximum Likelihood . . . . . . . . . . . 12
  • 12.2.1 Delta Method . . . . . . . . . . . 12
  • 12.3 Multiparameter Models . . . . . . . . . 12
  • 12.3.1 Multiparameter delta method . . 13
  • 12.4 Parametric Bootstrap . . . . . . . . . . 13

13 Hypothesis Testing 13

14 Bayesian Inference 14

  • 14.1 Credible Intervals . . . . . . . . . . . . . 14
  • 14.2 Function of parameters . . . . . . . . . . 14
  • 14.3 Priors . . . . . . . . . . . . . . . . . . . 15
  • 14.3.1 Conjugate Priors . . . . . . . . . 15
  • 14.4 Bayesian Testing . . . . . . . . . . . . . 15

15 Exponential Family 16

16 Sampling Methods 16

  • 16.1 The Bootstrap . . . . . . . . . . . . . . 16
  • 16.1.1 Bootstrap Confidence Intervals . 16
  • 16.2 Rejection Sampling . . . . . . . . . . . . 17
  • 16.3 Importance Sampling . . . . . . . . . . . 17

17 Decision Theory 17

  • 17.1 Risk . . . . . . . . . . . . . . . . . . . . 17
  • 17.2 Admissibility . . . . . . . . . . . . . . . 17
  • 17.3 Bayes Rule . . . . . . . . . . . . . . . . 18
  • 17.4 Minimax Rules . . . . . . . . . . . . . . 18

18 Linear Regression 18

  • 18.1 Simple Linear Regression . . . . . . . . 18
  • 18.2 Prediction . . . . . . . . . . . . . . . . . 19
  • 18.3 Multiple Regression . . . . . . . . . . . 19
  • 18.4 Model Selection . . . . . . . . . . . . . . 19

19 Non-parametric Function Estimation 20

  • 19.1 Density Estimation . . . . . . . . . . . . 20
  • 19.1.1 Histograms . . . . . . . . . . . . 20
  • 19.1.2 Kernel Density Estimator (KDE) 21
  • 19.2 Non-parametric Regression . . . . . . . 21
  • 19.3 Smoothing Using Orthogonal Functions 21

20 Stochastic Processes 22

  • 20.1 Markov Chains . . . . . . . . . . . . . . 22
  • 20.2 Poisson Processes . . . . . . . . . . . . . 22

21 Time Series 23

  • 21.1 Stationary Time Series . . . . . . . . . . 23
  • 21.2 Estimation of Correlation . . . . . . . . 24
  • 21.3 Non-Stationary Time Series . . . . . . . 24
  • 21.3.1 Detrending . . . . . . . . . . . . 24
  • 21.4 ARIMA models . . . . . . . . . . . . . . 24
  • 21.4.1 Causality and Invertibility . . . . 25
  • 21.5 Spectral Analysis . . . . . . . . . . . . . 25

22 Math 26

  • 22.1 Gamma Function . . . . . . . . . . . . . 26
  • 22.2 Beta Function . . . . . . . . . . . . . . . 26
  • 22.3 Series . . . . . . . . . . . . . . . . . . . 27
  • 22.4 Combinatorics . . . . . . . . . . . . . . 27

To read the PDF version, click here. 

Related articles

DSC Resources

Popular Articles