Models of infectious diseases are increasingly used to design and evaluate infection control strategies. The infectious disease models describe (or approximate) a contagion process on a given network structure. The dynamical behaviour of such models is often rich and complex. In rare situations we have observations on the network structure and we are interested in the spread of infection over this network. An example is the referral network of patients between hospitals, which allows us to model the spread of resistant bacteria between hospitals. In most situations we have observations on the infected persons (time of symptom onset, place, demographics) but we have no information on the underlying network structure (the ‘background’). We are interested in interpreting these observations in a ‘background independent’ manner without making strong assumptions about the network structure. Our approach is to use the observed infection events to reconstruct the causal structure of the infection network. We discuss how we can use this approach to make sense of observations during emerging epidemics.