On this research we are studying the observation process of a social system. Part of this research can be found on my Doctoral Thesis, and some on a paper that has been submitted to Social Networks (and rejected). Further work is being done to get this published. Comments are appreciated.
Below are the referees comments.
Reviewers’ comments:
Reviewer #1: This is a methodological paper, using four different datasets to study the effects of the period of time during which a network is observed affects the structure of ties to be mapped. In its present form, the paper makes a rather trivial claim -just saying that there is some effect. It does not explore any further how duration of observation time affects network structure, nor how the effects in questions vary depending on different properties of the network (particularly directionality of ties, composition and size of the nodes set, and others, e.g. presence of indicators of tie strength other than time). Neither does it derive any methodological recommendations for research practice. Therefore as such, it is not suitable for publication in SON. However, some of its findings such as how key network metrics (such as transitivity and betweenness) vary with observation time are interesting and may be developed further. In more detail:
The authors study the effects of aggregating connections over time. Aggregations are obtained by adding the weight of edges observed at different moments in time -for instance, a network observed over one hour will be the sum of what is observed in the first and in the second half-hours. While this method is acceptable, the authors never explain why it would be interesting to apply it at all, and why SON readers should care. In my view, a potential justification could be that in many cases, it is practically impossible to observe a system at very short intervals of time (say, every minute) and only longer-duration measures (say, every hour) are available: it may thus be useful to know which biases we might encounter in these cases. The databases of the authors allow to see the same system at both short and long time duration intervals and therefore, can potentially tell us something general about the differences between them. This insight can be used to infer the biases that
might be present when only long-duration data are available.
The authors use different datasets, but do not look deeper into their properties and therefore, their analyses remain largely superficial. For example, on the co-authorship network observed over ten years for two Universities: they do not consider that faculty members in either institutions must certainly have changed during that period of time owing to retirement, new hires, career moves etc. Therefore, what is the reference composition of the node set? Is it the total number of members who were present at least once between 2000 and 2009? Or the number of those who were present at all observations? Whether the authors take the one or the other set as a reference, is likely to have an effect on their results. They should therefore be clearer on their choices (and possibly compare the different options).
Similarly, the authors do not discuss similarities and differences between the datasets they refer to and therefore, their choice of data seems somewhat arbitrary (and the multiplication of datasets redundant). In fact, differences across datasets may have an impact on the very effect they aim to study and hence, need to be explored in greater depth. For example, does tie directionality play any role in determining the effects of length of observation? The co-authorship ties can safely be taken as undirected, but the student’s networks observed in two University courses through questionnaires are likely directional -the fact that A names B does not per se imply that B will name A. I am sure SON readers would like to know more about any effects of tie directionality. Node set size might also matter: there is in this sense a major difference between the two co-authorship networks, and one wonders what its effects might be, if any.
The choice of network metrics on which to measure the effects of longer observation times looks somewhat arbitrary. It is true that transitivity, degree, betweenness, and density are important and widely used measures, but why would they be more interesting here than, say, reciprocity or balance? The authors should either look at a broader range of measures or better justify their choice of focusing on just a few of them (ideally with the help of social network literature showing how, when, and for what social issues they are useful).
More generally, this seems to be a paper written by engineers or computer scientists who have relatively little knowledge of the social science literature on networks, its key insight, theoretical and methodological issues, and current debates. Since it is this community that reads SON, the authors should do a little more effort to better understand the expectations of social network researchers and adapt their presentation in order for it to speak to readers in a way that they can understand and appreciate. In particular, it would be useful to consider a bit more carefully what the relevant time horizons are in the different data sets, and how this should be interpreted (years or months in the co-authorship network, hours or days or week in the students’ network, and even shorter measures in Salathe’s school network).
Some minor points: the referencing style is not the one currently in use at SON and needs to be revised. The figures are neither numbered nor labelled; I have assumed that the order in which they appear correspond to their number but the lack of any explanations makes them very difficult to understand (what the red line indicates with respect to the black one etc.). The use of language will need some check before final publication.
Reviewer #2: In this article, the authors propose that researchers must consider how time intervals in network research can influence results. I think that this is an important issue worthy of discussion. However, I think that the authors need to extend their analysis and discussion to go further than just identifying the issue. This issue has been recognized by others, so perhaps future iterations of this paper can propose theoretical extensions to existing work. For example, the authors might address Moody’s (2002) discussion of the importance of time in diffusion studies and propose theoretical reasons for also considering time in affiliation studies (e.g., co-authorship) or state-based studies (e.g, friendship).
My core suggestion is that to make a solid contribution I believe that the authors will need to focus on theoretical decisions involved with time cutoffs and perhaps propose some guidelines for methodological choices. My feeling is that the current version of the article spends too much time proposing that time cutoffs influence results (which is interesting, but not novel). To me, the potential contributions begin on page 14 with the question: “how should we aggregate this (sp) connections?” Perhaps your data analysis of proximity interactions, not thoroughly discussed in the current piece, has the potential to provide theoretical (and methodological) answers to these questions.
In summary, I believe that the authors address an interesting point but do not develop their ideas beyond observing that time influences outputs. I appreciate that the authors included four empirical examples from various settings, but think that it would help to extend the discussion of why time is so important in each of these settings. Again, I suggest that the authors focus more on the theory than on the observation that time matters (e.g., in addition to explaining that statistics vary over time, I think that you need to say why it is theoretically important to recognize these changes). I hope that these comments are helpful.
Below are some potentially useful references that will inform your thinking on this issue:
*Moody, J. 2002. The importance of relationship timing for diffusion. Social Forces. Vol. 81(1), 25-66. Moody also has more recent work addressing longitudinal network analysis.
*Borgatti, S.P. and Halgin, D.S. 2011. On Network Theory. Organization Science. Borgatti’s typology of ties might provide theoretical insights on when and how time cutoffs influence network research. For example, temporal decisions are clearly very important in social network research of interactions (e.g., number of emails sent, number of sexual encounters, etc.) but perhaps less so in longitudinal analysis of states (e.g., friendship. Other states such as kinship are clearly static relations).
