^{1}

^{*}

^{†}

^{2}

^{3}

^{*}

^{†}

^{1}

^{†}

^{1}

^{2}

^{3}

Edited by: Nicole C. Kleinstreuer, National Institute of Environmental Health Sciences (NIEHS), United States

Reviewed by: Emilio Benfenati, Istituto Di Ricerche Farmacologiche Mario Negri, Italy; Cynthia Rider, National Toxicology Program Division (NIEHS), United States

*Correspondence: Simone Lederer,

This article was submitted to Predictive Toxicology, a section of the journal Frontiers in Pharmacology

^{†}ORCID: Tjeerd MH Dijkstra,

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

In synergy studies, one focuses on compound combinations that promise a synergistic or antagonistic effect. With the help of high-throughput techniques, a huge amount of compound combinations can be screened and filtered for suitable candidates for a more detailed analysis. Those promising candidates are chosen based on the deviance between a measured response and an expected non-interactive response. A non-interactive response is based on a principle of no interaction, such as Loewe Additivity or Bliss Independence. In a previous study, we introduced, an explicit formulation of the hitherto implicitly defined Loewe Additivity, the so-called Explicit Mean Equation. In the current study we show that this Explicit Mean Equation outperforms the original implicit formulation of Loewe Additivity and Bliss Independence when measuring synergy in terms of the deviance between measured and expected response, called the lack-of-fit. Further, we show that computing synergy as lack-of-fit outperforms a parametric approach. We show this on two datasets of compound combinations that are categorized into synergistic, non-interactive, and antagonistic.

When combining a substance with other substances, one is generally interested in interaction effects. Those interaction effects are usually described as synergistic or antagonistic, dependent on whether the interaction is positive, resulting in greater effects than expected, or negative, resulting in smaller effects than expected. From data generated with high-throughput techniques, one is confronted with massive compound interaction screens. From those screens, one needs to filter for interesting candidates that exhibit an interaction effect. To quickly scan all interactions, a simple measure is needed. Based on that preprocessing scan, those filtered combination candidates can then be examined in greater detail. In such a quick scan, one focuses uniquely on the measured response and not on possible mechanisms of action of each compound.

To determine whether a combination of substances exhibits an interaction effect, it is crucial to determine a non-interactive effect. Only when deviance from that so-called null reference is observed, can one speak of an interactive effect (

Independent of the indecisive opinions about the null reference, there are multiple proposals regarding how synergy can be measured given a null reference model. Many models are based on the concept of isoboles (

In this paper we measure synergy as the deviation over the entire response surface. One way to do so is the Combenefit method by measuring synergy in terms of volume between the expected and measured effect (

There is an increase in theoretical approaches to synergy, such as the recently re-discovered Hand model (

As the research area of synergy evolved from different disciplines, different terminologies are in common use. While in pharmacology, one refers to the Loewe model, in toxicology, the same principle is called concentration addition. The response can be measured among others in growth rate, survival, or death. It is usually referred to as the measured or phenotype effect or as cell survival. In this study we interchange the terms response and effect.

When measuring a compound combination, one also measures each agent individually. The dose or concentration is typically some biological compound per unit of weight when using animal or plant models or per unit of volume when using a cell-based assay. However, it can also be an agent of a different type for example a dose of radiation as used in modern combination therapies for cancer (

In

Before one can decide whether a compound combination exhibits a synergistic effect, one needs to decide on the expected effect assuming no interaction between the compounds. Such so-called null reference models are constructed from the conditional (mono-therapeutic) dose–response curves of each of the compounds, which we denote by _{j}_{j}_{1}, _{2}) such that

and

Thus, the conditional response curves are the boundary conditions of the null reference surface. For this study, we focus on Hill curves to model the conditional dose–responses. More detailed information can be found in

Loewe Additivity builds on the concepts of sham combination and dose equivalence. The first concept is the idea that a compound does not interact with itself. The latter concept assumes that both compounds that reach the same effect can be interchanged. Therefore, any linear combination of fractions of those doses which reach the effect individually and, summed up, are equal to one, yields that exact same effect. Mathematically speaking, if dose _{1}, _{2}), for which

holds, should yield the same effect as

where _{1}, _{2}) is implicitly given in Eq. 4. In the following we use the mathematical notation for the General Isobole Equation _{GI} (_{1}, _{2}) =

It was shown by

where _{1} of compound one to reach the same effect of compound two with dose _{2} (see _{1}, _{2}) is expressed as the effect of either one compound to reach that same effect. Under the LACC, all three models, Eq. 4, Eq. 5, and Eq. 6 are equivalent. It was further shown, that, in order for the LACC to hold, conditional dose–response curves must be proportional to each other, i.e. being parallel shifted on the _{2→1} (_{1}, _{2}) and _{1→2} (_{1}, _{2}) is synergistic or antagonistic and hence should be treated as non-interactive. We refer to the envelope spanned between the two explicit surfaces _{2→1} (_{1}, _{2}) and _{1→2} (_{1}, _{2}) in Eq. 5 and Eq. 6 as _{geary}. In contrast to that, in an effort to take advantage of the explicit formulation and to counteract the different behavior of Eq. 5 and Eq. 6 in case of a violated LACC,

A more extensive overview of Loewe Additivity and definition of null reference models together with visualizations can be found in

Bliss Independence assumes independent sites of action of the two compounds and was introduced a decade later than Loewe Additivity in (

where _{1} (_{i}_{i}_{i}

Here, we measure the effect in terms of cell survival or growth inhibition. Therefore the conditional response curves are monotonically decreasing for increasing concentrations or doses.

The records are normalized to the response at _{1} = 0, _{2} = 0, thus _{1}(0) = _{2}(0) = 1. To arrive from Eq. 8 to Eq. 9, one replaces any

While there are many mathematical variations to the general concept of Loewe Additivity (here we introduced five null-reference models based on this methodology), there is generally only one way to compute Bliss Independence.

The six models introduced in the previous section are null reference models in that they predict a response surface in the absence of compound interaction. We capture synergy in a single parameter to facilitate the screening process. This is different from other approaches, such as

Here, we measure synergy in two different ways, namely in fitting parametrized models or computing the lack-of-fit. The first method fits null reference models that are extended with a synergy parameter α. For these parametrized models α is computed by minimizing the square deviation between the measured response and the response spanned by the α-dependent model. For the second method the difference between a null reference model and the data is computed. For this method, which we refer to as lack-of-fit, the synergy score γ is defined as the volume that is spanned between the null reference model and the measured response.

Just as the conditional responses form the boundary condition for the null-reference surface (Eq. 1, Eq. 2), we want the conditional responses to be the boundary condition for all values of α. Explicitly, assuming a synergy model dependent on α is denoted by _{1}, _{2}ǀ α), then

with _{i}

We extend the six null reference models introduced in

Berenbaum originally equated the left-hand side of Eq. 4 to the so-called Combination Index _{CI} (_{1}_{2}ǀ

Note that this model violates the Synergy Desideratum in Eq. 11 as α not zero leads to deviations from the conditional responses. Explicitly, _{CI} (_{1}, 0ǀα) = _{1}((1 − α) _{1}) ≠ _{1}(_{1}). Although the Combination Index model violates the Synergy Desideratum, in practice it performs quite well and is in widespread use.

The explicit formulations in Eq. 5 and Eq. 6 are equivalent to the General Isobole Equation, _{GI} (_{1}, _{2}), given in Eq. 4, under the LACC (

With this, we can extend the Explicit Mean Equation model _{mean} (_{1}, _{2}) in Eq. 7 to a parametrized synergy model:

which we refer to as _{mean} (_{1}, _{2}ǀ α). As _{2→1} (_{1}, _{2}ǀ α) and _{1→2} (_{1}, _{2}ǀ α) do not fulfill the Synergy Desideratum, _{mean} (_{1}, _{2}ǀ α) also does not fulfill it.

To investigate the difference between the two models _{2→1} (_{1}, _{2}) (Eq. 5) and _{1→2} (_{1}, _{2}) (Eq. 6) we treat compound one and two based on the difference in slopes in the conditional responses (for more detailed information on the different parameters in Hill curves, refer to _{large→small} (_{1}, _{2}ǀ α) and _{small→large} (_{1}, _{2}ǀ α).

Analogously, we extend the Geary model _{geary} (_{1}, _{2}) to a synergy model and refer to it as _{geary} (_{1}, _{2}ǀ α). Based on a comment of _{2→1} (_{1}, _{2}) and _{1→2} (_{1}, _{2}) yield the same surface under the LACC but do rarely in practice. Therefore, it cannot be determined whether a response that lies between the two surfaces is synergistic or antagonistic and hence should be treated as non-interactive. Thus, if α from _{1→2} (_{1}, _{2}ǀ α) and α from _{2→1} (_{1}, _{2}ǀ α) are of equal sign, the synergy score of that model is computed as the mean of those two parameters. In case the two synergy parameters are of opposite sign, the synergy score is set to 0:

Next, to extend the null reference model following the principle of Bliss Independence, we extend Eq. 8 to

The motivation for this model is that any interaction between the two compounds is caught in the interaction term of the two conditional responses. In case of no interaction, the synergy parameter α = 0, which leads to (1 + α) = 1, and results in no deviance from the null reference model. As we use the formulation of Eq. 9 due to measuring the effect as survival, we reformulate Eq. 17 analogously as we did to get from Eq. 8 to Eq. 9: by replacing _{i}_{i}_{i} (_{i}) Hence, Eq. 17 takes the form:

This model does satisfy the requirement of no influence of the synergy parameter on conditional doses: _{bliss} (_{1}, 0ǀ α) = _{1} (_{1}) and _{bliss} (0, _{2}ǀ α) = _{2} (_{2}) as _{i}_{bliss} (_{1}, _{2})→ − ∞ if α→−∞. The same holds for the formulations of Loewe Additivity. The implicit formulation becomes impossible to match and for the explicit formulations, the dose expression within brackets of _{2→1} (_{1}, _{2}ǀ α) becomes negative. Additionally, α > 1 is not possible for _{CI} (_{1}, _{2}ǀ α), as the left-hand side of Eq. 12 can not be negative. Such behavior is also known from other models, e.g. for the Greco flagship model for negative synergy scores (

Despite of the Synergy Desideratum being violated for the models that build up on the Loewe Additivity principle, there is no further effect on the model comparison presented in

The second method to measure synergy investigated here is to compute the lack-of-fit of the measured response of a combination of compounds to the response of a null reference model derived from the conditional responses. We refer to this synergy value as γ:

with _{1}, _{2} ǀ ϴ) the estimated effect with parameters ϴ of the fitted conditional responses following any non-interactive model and _{1}, _{2}). This method was used in the AstraZeneca DREAM challenge (

In all, we have introduced six null reference models, five of them building up on the concept of Loewe Additivity and one on Bliss Independence. We further have introduced two methods to compute synergy, the parametric one and the lack-of-fit method, where both synergy parameters α and γ are positive if the record is synergistic, negative, if antagonistic. This results in 12 synergy model–method combinations: the parametric ones, _{CI} (_{1}, _{2}ǀ α) (Eq. 12), _{large→small} (_{1}, _{2}ǀα), and _{small→large} (_{1}, _{2}ǀα) (Eq. 13, Eq. 14, dependent on the slope parameters) together with their mean, _{mean} (_{1}, _{2}ǀ α) (Eq. 15), _{geary} (_{1}, _{2}ǀ α) (computation of α_{geary} explained in Eq. 16) and _{bliss} (_{1}, _{2}ǀ α) (Eq. 17). For the lack-of-fit method, we take as the null reference: _{GI} (_{1}, _{2}) (Eq. 4), _{large→small} (_{1}, _{2}) and _{small→large} (_{1}, _{2}) (Eq. 5, Eq. 6), with the Explicit Mean Equation, _{mean}(_{1}, _{2}) (Eq. 7), _{geary} (_{1}, _{2}) (analogously to Eq. 16) and _{bliss} (_{1}_{2}) (Eq. 9).

Before applying the two methods presented in _{0}, the measured response at zero dose concentration from both compounds. Second, we discard outliers using the deviation from a spline approximation. Third, we fit both conditional responses of each record, namely the responses of each compound individually, to a pair of Hill curves (Eq. S1,

We apply the two different methods to calculate the synergy parameters α and γ to each record. First, for the parametrized synergy models, we apply a grid search for α, for ^{th}^{(i)}^{th}

Note that we exclude the conditional responses that we used to fit Θ from the minimization. Second, we apply the lack-of-fit method from _{geary} (_{1}, _{2}) model, we compute the integral over all data points for which the difference between expected effect or _{2→1} (_{1}, _{2}) or _{1→2} (_{1}, _{2}) and the measured effect are of the same sign. If they are of opposite sign, the difference is set to zero. In

Description of the analysis steps of the lack-of-fit method for the compound pair TER and STA from the Cokol dataset. This compound pair is categorized as synergistic according to

To evaluate the two methods introduced in

The Mathews Griner dataset is a cancer compound synergy study by

The Cokol dataset comes from a study about fungal cell growth of the yeast

Based on the longest arc length of an isobole that is compared to the expected longest linear isobole in a non-interactive scenario, where Loewe Additivity serves as null reference model, each record was given a score. In more detail, from the estimated surface of a record assuming no interaction, the longest contour line is measured in terms of its length and direction (convex or concave). A convex contour line leads to the categorization of a record as synergistic and the arc length of the longest contour line determines the strength of synergy. A concave contour line results in an antagonistic categorization with its extent being measured again as the length of the longest isobole. Thus the Cokol dataset not only comes with a classification but also with a synergy score similar to α or γ.

To our knowledge, these two datasets are the only high-throughput ones with a classification into the three synergy classes: antagonistic, non-interactive and synergistic. Both datasets are somewhat imbalanced because interactions are rare (

Number of cases categorized as synergistic, antagonistic or non-interactive in the two datasets Mathews Griner and Cokol.

Synergistic | Antagonistic | Non-interactive | |
---|---|---|---|

Mathews Griner | 121 | 90 | 252 |

Cokol | 50 | 68 | 82 |

Using the two methods of computing the synergy score, the parametric one (

Having computed the synergy scores α and γ from the two different methods as described in

To compare the parametric and lack-of-fit methods, we plot the correlation values as a scatter plot per method (see _{geary} (_{1},_{2}|α) and _{small→large} (_{1}_{2}|α) applied to the Cokol dataset. For both datasets, the highest correlation scores result from those null reference models that are based on the Loewe Additivity principle. The Bliss null reference model performs worst for the Mathews Griner set for both methods. For the Cokol data it is the second worst model. To a certain extent this can be explained due to the classification of the Cokol dataset being based on the isobole length relative to non-interactive isoboles, which is a Loewe Additivity type analysis. As the categorization of the Mathews Griner dataset is based on visual inspection, we cannot explain the bad performance of _{bliss} (_{1},_{2}) for that dataset. On both datasets, _{GI} (_{1},_{2}), _{large→small} (_{1},_{2}) and _{mean} (_{1},_{2}) perform best for the lack-of-fit method. For the Mathews Griner dataset, _{large→small} (_{1},_{2}) dominates marginally over the General Isobole Equation and Explicit Mean Equation model. For the Cokol dataset, the Explicit Mean Equation dominates for both methods.

Scatter plot of Kendall rank correlation coefficient for both datasets, Mathews Griner (left) and Cokol (right). The Kendall correlation measures the rank correlation of the original categorization and the computed synergy scores. The higher the correlation, the more similar the score ranking. The correlation values from the synergy scores α, computed with the parametric approach, are plotted on the

To further investigate the performance of the methods and null reference models, we plot the synergy scores of the best performing models based on the Kendall rank correlation coefficient analysis (

In

Computed synergy scores γ of the Cokol data of the best models according to the Kendall in rank correlation coefficient and the receiver operating characteristic (ROC) analysis in

Scatter plot of synergy scores γ of the Mathews Griner dataset. The scores are computed with the lack-of-fit method. Displayed are the four best models according to the Kendall rank correlation coefficient and ROC analysis in _{bliss}(_{1},_{2}) show lowest correlation (first three cases). There is a large difference between the correlation between the additive models and the comparison of Bliss Independence by roughly 0.3.

For the other three models depicted in

We further looked in detail into dose combinations for which both the _{GI} (_{1},_{2}) and _{mean} (_{1},_{2}) yield positive synergy values for antagonistic cases and into dose combinations for which the _{mean} (_{1},_{2}) model results in negative synergy values for records which are labeled as synergistic. In total we found four dose combinations. A visualization of the observed and expected responses based on the Explicit Mean Equation model is shown in _{∞} being above 0.65 (comp. right panel of

We looked up the three dose combinations (excluding the one where the compound is combined with itself) in the Connectivity Map (

In _{mean} (_{1},_{2}) and _{large→small} (_{1},_{2}). The coloring of the scores is based on the original categorization as antagonistic, non-interactive or synergistic as provided by

In _{large→small} (_{1},_{2}) achieve higher values than those from the other two Loewe Additivity based models. This becomes obvious when comparing the null-reference surfaces of those three models, as depicted in _{large→small} (_{1},_{2}) spans a surface above those surfaces spanned by Explicit Mean Equation or General Isobole Equation. Therefore, in synergistic cases where the measured effect is greater, and hence the response in cell survival smaller, the difference from the null-reference surface to _{large→small} (_{1},_{2}) is greater than to the other two models. We suspect the synergy models from the Cokol dataset to be better separable due to the experimental design of the dataset. All compounds were applied up to their known maximal effect dose. This was not the case for the Mathews Griner dataset, where all compounds were applied at the same fixed dose range.

Scatter plot of synergy scores γ of the Cokol dataset. The scores are computed with the lack-of-fit method. Displayed are the four best models according to the Kendall rank correlation coefficient and ROC analysis in _{mean}(_{1}, _{2}) and _{GI}(_{1}, _{2}) show highest correlation (center right), _{bliss}(_{1}, _{2}) shows lowest (first three comparison cases).

All in all, the lack-of-fit method performs better for any model when applied to the Mathews Griner dataset and mostly better for the Cokol dataset, with the exception of the _{small→large} (_{1},_{2}) and Geary model. We suggest, that the lack-of-fit should be preferred over the parametric method, due to the overall performance on both datasets. When using the lack-of-fit method, the Explicit Mean Equation model performs either second best (Mathews Griner dataset), or best (Cokol dataset). The other two well performing models, the explicit _{large→small} (_{1},_{2}) or the original implicit formulation of Loewe Additivity, the General Isobole Equation, do not perform equally well on both datasets. To exclude any bias from these models for different datasets, the Explicit Mean Equation should be preferred.

The rise of high-throughput methods in recent years allows for massive screening of compound combinations. With the increase of data, there is an urge to develop methods that allow for reliable filtering of promising combinations. Additionally, the recent success of a synergy study of

In this study we use two datasets of compound combinations that come with a categorization into synergistic, non-interactive or antagonistic for each record.

Based on the fitted conditional responses, we compute the synergy scores of all records. We compare six models that build on the principles of Loewe Additivity and Bliss Independence. Those six models are used with two different methods to compute a synergy score for each record. The first method is a parametric approach and is motivated by the Combination Index introduced by

We compare the computed synergy scores from both methods, each applied with the six reference models, based on Kendall rank correlation coefficients. Based on these correlation coefficients we investigate the reconstruction of ranking of the records (see

For the above comparison of the six null reference models and the two methods, we rely on the underlying categorization of both datasets. All performance metrics are based on how well the predicted synergy scores agree with the underlying categorization. The categorizations of both datasets were created very differently from one another. On one hand, the Mathews Griner dataset was categorized on a visual inspection, on account of which we cannot be certain about the assumptions made that guided the decision making process. On the other hand, the categorization of the Cokol dataset is based on the principle of Loewe Additivity. This leads to the natural preference of null models that are based on Loewe Additivity over those based on Bliss Independence, which we find back in our analysis. Irrespective of the origin of the classification, we stress that the labels were provided to us by independent researchers and hence were not biased in any way to favor the Explicit Mean Equation model.

Note that we conduct the research only on combinations of two compounds. Research on higher-order combinations is usually performed with the principle of Bliss Independence (

Meanwhile, it is shown in

The comparison of the parametric method with the lack-of-fit method shows a superiority of the lack-of-fit method. To recall, the motivation behind the parametric approach was the statistical advantages of such an approach. It allows to define an interval around α = 0 in which a compound combination can be considered additive. For the lack-of-fit method, such statistical evaluation can not be done directly, but could be performed on the basis of bootstrapping.

_{1} and _{2} that are assumed to reach a fixed effect

Finally, to asses how distinguishable the synergy scores γ are, we visualize the synergy scores based on the underlying category (

During the analysis, we observed higher synergy scores when applying the Bliss Independence principle as null reference model. This is due to the more conservative null reference surface as derived from Bliss Independence [see exemplary comparison of isoboles from most of the models discussed here in (

Maximal response _{∞} (left) and slope parameters _{mean}(_{1}, _{2}) and _{GI}(_{1}, _{2}) in a synergy score of opposite sign to its categorization from the Cokol dataset. Different records are depicted in different colours. The original categorization of each record is depicted per shape. The conditional responses of one record, and hence their Hill curve parameters, are grouped depending on size of the Hill curve parameter

We want to emphasize the performance benefit of the recently introduced Explicit Mean Equation (

Although the performance of models and methods are consistent across the two (quite different) datasets considered in this study, reliable comparison of different models and methods would benefit from the availability of drug screening datasets that are available with ground truth labeling.

The datasets analyzed in this manuscript are not publicly available. Requests to access the datasets should be directed to Bhagwan Yadav,

TD conceived the research. SL performed the data analysis. All authors wrote or contributed to the writing of the manuscript.

This work was supported by the Radboud University and CogIMon H2020 ICT-644727. We acknowledge institutional funding from the Max Planck Society.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

We thank Bhagwan Yadav for the sharing of the code used for the analysis in

The Supplementary Material for this article can be found online at: