NeuroBayes A modern analysis tool from High Energy Physics and its way as prognosis tool into business Prof. Dr. Michael Feindt CETA - Centrum für Elementarteilchen- und Astroteilchenphysik Universität Karlsruhe Phi-T GmbH, Karlsruhe Physikalisches Kolloquium Universität Siegen, June 21, 2007 Predictable The result of simple classical physics processes is exactly predictable (one cause leads to one definite unique result, determinism) Examples: pendulum, planets, billard, electromagnetism
Unpredictable Purely random processes are not predictable at all (even if the initial conditions are completely known!) Examples: Lottery (Too many tiny influences and branchings, deterministic chaos) radioactive decay (quantum mechanics) electronic noise Probability Many systems in nature and life: Mixture of predictable and unpredictable (quasi-) random or chaotic components. Probability statements, statistics.
Today s topic: Extraction of a predictable component from empirical data (or Monte Carlo simulations) Statistically relevant predictions for future events individual event Individualisisation of probability statements: mean event conditional probabilities: f(t x), dependent on individual event with properties x instead of general (a priori) probability f(t) Particle physics experiments Many 1.000.000.000 identical experiments for many years: Collisions of e.g. electrons and positrons (at LEP)
Quantum Mechanics: In every collision something else happens! Experiments: Observe mean values, distributions, correlations, determine parameters (mean lifetime, spin, parity etc) from that. OPAL experiment at LEP NeuroBayes task 1: Classifications Classification: Binary targets: Each single outcome will be yes or no NeuroBayes output is the probability that answer is yes. Examples: > This elementary particle is a K meson. > Germany will become soccer world champion in 2006. now P=0 > Customer Meier will have liquidity problems in the next year. > Customer Müller will buy this product. > This equity price will rise.
NeuroBayes task 2: Conditional probability densities Probability density for real valued targets: For each possible (real) value a probability (density) is given. From that all statistical quantities like mean value, median, mode, standard deviation, percentiles etc can be deduced. Examples: > Energy of an elementary particle > Price change of an equity or option > Company turnaround or earnings Classification == Hypothesis testing Cut in a 1-dimensional real test-statistic which is correlated to the probability of hypothesis H0: Accept hypothesis H0, if t<t(cut) Error of 1. kind: P1(true hypothesis will be rejected) Error of 2. kind: P2(wrong hypothesis is accepted)
Hypothesis testing 0.7 0.8 0.9 1.0 A statistical method is the better the nearer it reaches the point (1,1) in the purity-efficiency-plot Different cuts in in tt Optimal choice of working point according to particular task: How does the total error of the analysis scale with ε und P? Optimal working point Determining the working point (scan through cuts on network output)
Construction of a test statistic: How to make 100 dimensions one real number Neural networks Neural networks: Self learning procedures, copied from nature Frontal Lobe Motor Cortex Parietal Cortex Temporal Lobe Brain Stem Occipital Lobe Cerebellum
Neural networks The information (the knowledge, the expertise) is coded in the connections between the neurons Each neuron performs fuzzy decisions A neural network can learn from examples NeuroBayes principle Input Preprocessing NeuroBayes Teacher: Learning of complex relationships from existing data bases NeuroBayes Expert: Prognosis for unknown data Significance control Postprocessing Output
How it works: training and application Historic or simulated data Data set a =... b =... c =...... t =! NeuroBayes Teacher Actual (new real) data Expert system Expertise Probability that hypothesis is correct (classification) or probability density for variable t Data set a =... b =... c =...... t =? NeuroBayes Expert f t t Naïve neural networks and criticizm We vetriedthatbutitdidn tgivegood results - Stuck in local minimum -Learningnotrobust We ve tried that but it was worse than our 100 person-years analytical high tech algorithm - Selected too naive input variables - Use your fancy algorithm as INPUT! We ve tried that but the predictions were wrong - Overtraining: the net learned statistical fluctuations Yeah but how can you estimate systematic errors? - How can you with cuts when variables are correlated? - Tests on data, data/mc agreement etc possible and done.
Address all these topics and build a professional robust and flexible neural network package for physics, insurance, bank and industry applications: NeuroBayes <phi-t>: Foundation out of University of Karlsruhe, sponsored by exist-seed-programme of the federal ministery for Education and Research BMBF History 2000-2002 NeuroBayes -specialisation for economy at the University of Karlsruhe Oct. 2002: GmbH founded, first industrial projects June 2003: Removal into new office 199 qm IT-Portal Karlsruhe Exclusive rights for NeuroBayes Staff all physicists (almost all from HEP) Customers (among others): BGV and VKB car insurances AXA and Central health insurances Lupus Alpha Asset Management dm drogerie markt (drugstore chain) Otto Versand (mail order business) Libri (book wholesale) Thyssen Krupp (steel industry)
<phi-t> NeuroBayes > is based on neural 2nd generation algorithms, Bayesian regularisation, optimised preprocessing with transformations and decorrelation of input variables and linear correlation to output. > learns extremely fast due to 2nd order methods > is extremly robust against outliers > is immune against learning by heart statistical noise > tells you if there is nothing relevant to be learned > delivers sensible prognoses already with small statistics > can make binary decisions (classification) > can predict complete probability densities
Bayesian Regularisation Use Bayesian arguments to regularise network learning: Likelihood Prior Posterior Evidence Learn only statistically relevant information, suppress statistical noise
input variables ordered by relevance (standard deviations of additional information) Ramler-plot (extended correlation matrix)
Ramler-II-plot (visualize correlation to target) Visualisation of single input-variables
Visualisation of correlation matrix Variable 1: Training target Visualisation of network performance Purity vs. efficiency Signal-effiziency vs. total efficiency (Lift chart)
Visualisation of NeuroBayes network topology
Conditional probability density reconstruction: Aim: Aim: r Bayesian Bayesian estimator estimator f ( t x) for for a a single single multidimensional multidimensional measurement measurement x r.. x r x r "Components "Components of of may may be be correlated. correlated. "Components "Components of of should should be be correlated correlated to to t t or or its its uncertainty. uncertainty. "All "All this this should should be be learned learned automatically automatically in in a a robust robust way way from from data data bases bases "containing "containing Monte-Carlo Monte-Carlo simulations simulations or or historical historical data. data. Note: Note: Conditional Conditional probability probability density density contains contains much much more more information information than than just just the the mean mean value, value, which which is is determined determined in in a a regression regression analysis. analysis. It It also also tells tells us us something something about about the the uncertainty uncertainty and and the the form form of of the the distribution, distribution, in in particular particular non-gaussian non-gaussian tails. tails.
Conditional probability densities in particle physics What is the probability density of the true B momentum in this semileptonic B candidate event taken with the CDF II detector with these n tracks with those momenta and rapidities in the hemisphere, which are forming this secondary vertex with this decay length and probability, this invariant mass and transverse momentum, this lepton information, this missing transverse momentum, this difference in Phi and Theta between momentum sum and vertex topology, etc pp t x r r f ( t x) Prediction of the complete probability distribution event by event unfolding - r f ( t x) Expectation value Standard deviation volatility Mode Deviations from normal distribution, e.g. crash probability t
Physics Research Examples I Classification: Hadron Identification (DELPHI at CERN): Doubled signal strength at constant background level by neural network classification original method : several 10 millions CHF cost NeuroBayes predecessor: Additional factor of 2 with very limited additional effort
NeuroBayes soft electron identification for forcdf II II (Ulrich Kerzel, Michael Milnik, M.F.) Thesis U. U. Kerzel: on on basis of of Soft Electron Collection (much more efficient than cut selection or orjetnet with same inputs - after clever preprocessing by byhand and careful learning parameter choice this could also be beas as good as as NeuroBayes Physics Research Examples II Classification: electron identification at CDF II (Fermilab, USA) J/psi to electron signal increased by 135% by NeuroBayes --> > Efficiency gain for single electron identification 53%
Physics Research Examples III (DELPHI, CERN) Optimised reconstruction of real valued quantities: extended regression much improved resolution (narrow peak around +-0) + by NeuroBayes-technology Physics Research Examples IV (DELPHI, CERN) Resolution of azimuthal angle of inclusively Resolution of azimuthal angle of inclusively reconstructed B-hadrons in the DELPHIdetector reconstructed B-hadrons in the DELPHIdetector first neural reconstruction of a direction first neural reconstruction of a direction NeuroBayes NeuroBayes phi-direction phi-direction Best Best classical" classical" chi**2- chi**2- fit fit (BSAURUS) (BSAURUS) After selection cut on estimated error: No No selection: After selection cut on estimated error: selection: Resolution massively improved, no tails Improved Improved resolution Resolution massively improved, no tails resolution ==> allows reliable selection of good events ==> allows reliable selection of good events
Hadron collider: No No good MC for forbackgrounds available MC for forresonance production with different J PC PC assumptions Idea: take background from sidebands in in data check that network cannot learn mass X(3872) analysis: X soft NeuroBayes selection signal-like background-like hardly loose any signal
Hard NeuroBayes cut: Very clean X(3872) signal Training with weights: Neural Network Spin Parity Analysis Use data from sidebands as background sample Use phase space decay MC with modelled p T distribution as signal sample Calculate squared amplitudes for specific spin-parity assignments and use as weights in training Hard cut on neural network trained by correct hypothesis should increase signal over background
J PC PC =1 =1 (ππ) s hypothesis s Very hard NeuroBayes cuts: a not so so good hypothesis for forx J PC PC =1 =1 (ππ) ss hypothesis Very hard NeuroBayes cuts: A good hypothesis for forx CDF II II (also without NN): X(3872) can only be be1 ++ ++ (preferred) or or2 -+ -+ Decay into J/ψ ρisospin violating if ifsimple charmonium. X is isa good candidate for fora D^*D molecule (directly on on threshold)
Search for fororbitally excited B ss in in B + K -- (with B + J/ψ K) K) (Martin Heck, Michal Kreps, M.F.) NeuroBayes helps helps in in finding hitherto undiscovered hadrons Clear Clear evidence for for two two peaks Stability against network cuts cuts proven Making MC for forhadronic background without specific model: Multidimensional correlated regression using NeuroBayes Use data in non-resonance region as signal Use phase space MC as background Train NeuroBayes network, NN output O is Bayesian a posteriori probability that event stems from signal (i.e. data distribution) rather than phase space MC: O=P(S) with P(S)+P(B)=1 Calculate weight W= P(S)/P(B) = O/(1-O) Phase space MC events with this weight W look like data! MC modelling of complicated background is possible! Opens new roads for likelihood fits
Some kinematical variable distributions (J/ψ π + π - selection) Black: real data Red: weighted phase space MC NeuroBayes: Increase of of efficiency at at same purity as as best cut based selection: Improve significance from 18.4 to to 22.1 Optimisation of of B ss reconstruction for forb ss oscillation analysis
Presse (Die Welt vom 21. April 2006) Last years s sensation is this year s calibration ( and next years background) Univ. Karlsruhe: Development of of complex b flavour tagger for forhadron colliders using a network of of NeuroBayes networks :: Improves statistical power by byfactor 2!! Calibration of of New Flavor Tagging Algorithms using B ss Oscillations Ph.D. thesis Ph. Ph. Mack, July 2007
Some applications in high energy physics DELPHI (mainly predecessors of NeuroBayes) Kaon, proton, electron id Optimisation of resolutions inclusive B- E, φ, θ, Q-value B**, B s ** enrichment B fragmentation function Limit on B s -mixing B 0 -mixing B- F/B-asymmetry B-> wrongsigncharm CDF II: (work in Progress) Electron ID, muon ID, kaon/proton ID Optimisation of resonance reconstruction (X, Y, B s, B s **) Spin parity analysis, likelihood analyses B-Tagging for top, Higgs, etc. B-Flavour Tagging for mixing analyses (new combined tagging) Direct competition I Data-Mining-Cup 2005: Fraud recognition in internet trading 531 participants from 176 universities from 41 countries. 6 Karlsruhe students got Phi-T NeuroBayes and support in lecture and computer lab: They achieved the top positions 2,3,4,5,6,7! www.data-mining-cup.de
Direct competition II Data-Mining-Cup 2006: Award ceremony just last week. Predict whether ebay auctions will result in higher than average prices 579 participants from 177 universities from 42 countries. 11 Karlsruhe students got Phi-T NeuroBayes and support in lecture And computer lab: They achieved the top left to tight: Simon Honc, Moritz Schlie, Jens Salomon positions 1,2,3,4, 6,7, 9,10, 12, 16, 22! www.data-mining-cup.de Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Applications of NeuroBayes in Economy > Medicine and Pharma research (not yet ) e.g. effects and undesirable effects of drugs early tumor recognition > Banks e.g. credit-scoring (Basel II), finance time series prediction, valuation of derivates, risk minimised trading strategies, client valuation > Insurances e.g. risk and cost prediction for individual clients, probability of contract cancellation, fraud recognition, justice in tariffs > Trading chain stores: turnover prognosis for individual articles/stores Necessary prerequisite: Historic or simulated data must be available! Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 64
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Press Badische Neueste Nachrichten 17.November 2004 Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 65 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Presse 23. 8. 2005 Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 66
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Press Die Welt, 13. Sept. 2005 Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 67 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Presse BNN 20.9.2005 (dpa-meldung) Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 68
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Television Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 69 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Not only theory, but works in practical life : Just confirmed by practinioneer at insurance conference Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 70
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Risk analysis for a car insurance BGV Results for a the Badischen Gemeinde-Versicherungen: since May 2003: radically new tariff for young drivers! New variables added to calculation of the premium. Correlations taken into account. Risk und premium up to a factor of 3 apart from each other! Even probability distribution of costs can be predicted Premature contract cancellation also well predictable Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 71 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA The unjustice of insurance premiums Anzahl Kunden Ratio of the accident risk calculated using NeuroBayes to premium paid (normalised to same total premium sum): The majority of customers (with low risk) are paying too much. Less than half of the customers (with larger risk) do not pay enough, some by far not enough. These are currently subsidised by the more careful customers. Risiko/Prämie Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 72
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Prediction of contract cancellation for an insurance The prediction really holds: Test on a new statistic year Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 73 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Contract cancellations in a large financial institute Real cancellation rate as function of cancellation rate predicted by NeuroBayes Very good performance within statistical errors Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 74
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Chain stores turnaround predictions: Mittlerer Brutto-Umsatz pro Filiale Mittlerer Brutto-Umsatz pro Filiale Ostern Muttertag Valentinstag Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 75 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Results: Probability density for each shop and each day in the near future (up to ½ year) Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 76
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Near Future Turnaround Predictions for Chain Stores 1. Time series modelling 2. Correction and error estimate using NeuroBayes Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 77 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Turnover prognosis for mail order business Test at Germany s largest mail order business: Predict season s turnover of all articles/ colours/ sizes/ catalogues: 4 seasons for training, test on 2 following seasons Surprise for customer (and also us): Phi-T- prognoses mostly significantly better than estimates of (experienced) disponents. Optimal prognosis: NeuroBayesprognosis which also takes into account the (unbiased) disponents estimates as one input. (Subjective measures like fashionable, good-looking not in database.) Huge optimisation potential of many processes in the request/ supply chain! Individual determination of uncertainty allows optimisation by NewsBoy-model (relation to Black Scholes) Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 78
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 79 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Presse Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 80
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Prognosis of individual health costs Pilot project for a large private health insurance Prognosis of costs in following year for each person insured with confidence intervals 4 years of training, test on following year Results: Probability density for each customer/tarif combination Very good test results! Kunde N. 00000 Mann, 44 Tarif XYZ123 seit ca. 17 Jahre Has potential for a real and objective cost reduction in health management Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 81 Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA Prognosis of financial markets VDI-Nachrichten, 9.3.2007 NeuroBayes based risk averse market neutral fonds for institutional investors Lupus Alpha NeuroBayes Short Term Trading Fonds Test Test Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 82
Start Idee NeuroBayes Idee Hintergrund Ziele NeuroBayes f(t x) Beispiele Historie Anwendung Prinzip Funktion Beispiel Konkurrenz Projekt l Forschung Projekt ll Ablauf Spiel Summary A BA The <phi-t> mouse game: or: even your ``free will is predictable //www.phi-t.de/mousegame Michael Feindt NeuroBayes Kolloquium Siegen June 21, 2007 83 Documentation Basics: M. Feindt, A Neural Bayesian Estimator for Conditional Probability Densities, E-preprint-archive physics 0402093 M. Feindt, U. Kerzel, The NeuroBayes Neural Network Package, NIM A 559(2006) 190 Web Sites: www.phi-t.de (Company web site, German & English) www.neurobayes.de (English site on physics results with NeuroBayes) www-ekp.physik.uni-karlsruhe.de/~feindt (some NeuroBayes talks can be found here)