Providing guidance in the dark: rare renal diseases and the challenge to improve the quality of evidence
+ Author Affiliations
1CNR-IBIM, Istituto di Biomedicina, Epidemiologia Clinica e Fisiopatologia, delle Malattie Renali e dell’Ipertensione Arteriosa, Reggio Calabria, Italy
2Renal Division, Ghent University Hospital, Ghent, Belgium
- Correspondence and offprint requests to: Davide Bolignano; E-mail: firstname.lastname@example.org
↵* ERBP Methods Support Team.
- Received February 5, 2013.
- Accepted July 8, 2013.
Among renal diseases, over 100 conditions meet the epidemiological criteria to be defined as rare, including disorders in development, transport and metabolism. Clinical management of rare diseases is likely to be less investigated than that of common disorders and for this reason the scientific evidence to support clinical practice is limited. Furthermore, no specific and validated methods for designing, carrying out or analyzing clinical trials in small populations exist with important consequences for evidence-based medicine. In this paper we aim at discussing the inherent difficulty in finding evidence in rare renal diseases, providing some suggestions on how the quality of evidence and the guidance in these diseases can be improved.
In the USA, the National Institute of Health Office of Rare Diseases Research  defines as ‘rare’ a disease that affects <200 000 people. In Europe, a disease is considered rare when the prevalence is lower than 5 per 10 000 persons. It has been estimated that, so far, between 5 000 and 8 000 rare diseases have been characterized, which collectively affect 6–8% of the European population . Clinical management of rare diseases is inherently bound to be less investigated than that of common disorders and for this reason these diseases are often without specific treatment (orphan diseases). Rare diseases exist, such as nephrogenic diabetes insipidus and Fabry’s disease, where we have detailed pathophysiologic knowledge and also specific treatments. However these diseases are typically included among orphan diseases. The definition of orphan diseases has the broad scope of raising the attention of public and the scientific community on a growing series of rare diseases where knowledge on the pathophysiology and natural history is limited with little awareness among doctors and with no or very costly treatment. Approximately 80% of these conditions are genetic in nature. Progresses in genetics, from basic to clinical science, have opened unprecedented opportunities for the identification of faulty gene(s) responsible for these diseases thereby allowing the development of genetic tests and well-targeted treatments. Importantly, knowledge derived from the study of these disorders may also lead to a better understanding of normal biological processes as well as of other polygenic and acquired diseases. In this article, we discuss why the scientific evidence to support clinical practice is limited, what the consequences are for evidence-based medicine and how the quality of evidence and the guidance in these diseases can be improved.
RARE RENAL DISEASES
Among renal diseases, over 100 conditions meet the epidemiological criteria to be defined rare, including disorders in development, transport and metabolism . The list comprises the wide spectrum of polycystic kidney disorders, Fabry and Alport disease but also less known and more rare conditions (see Table 1). Some of these diseases selectively affect the kidney, while others may extend to other organs, either before or after renal involvement. In nephrology, rare diseases are mostly genetic so that the terms ‘inherited’ or ‘congenital’ are often used interchangeably with ‘rare’. Diseases caused by monogenic alterations are less common, often first expressed in childhood and associated with typical phenotypes and weak or no environmental influence. Polygenic disorders are more common, manifest later in life and are strongly influenced by a complex interaction between genetic and environmental factors. Although a single rare disorder of the kidney can hardly be considered to be a public health challenge, however, taken all together, rare renal diseases affect a significant number of individuals with important economic impacts on health services. Collectively, rare renal disorders rank worldwide as the fifth cause of end-stage kidney disease (ESKD) requiring renal replacement therapy after diabetes, hypertension, glomerulonephritides and pyelonephritis with an estimated total contribution of 10% to the annual incidence of ESKD . However, so far, only for few rare renal diseases have disease-specific international registries been established and publically available information on these disorders is often sparse and scarce. Providing evidence-based, balanced recommendations on diagnosis and management of these diseases is therefore a challenging, but necessary task for the renal community.
THE CONUNDRUM OF WEAK EVIDENCE IN RARE RENAL DISEASES
Observational studies represent a valid source of knowledge for addressing simple research questions in common as well as in rare diseases. Observational studies can provide key information on the epidemiology (e.g. incidence) and natural course of a given disease and suggest potential risk factors associated with different phenotypic manifestations of rare diseases. Because of sample size problems, cohort studies in rare diseases are tantalizing. Case–control studies in rare diseases are the only solution for diseases with a long latency period. However, this study design is weaker for assessing causal relationships than cohort studies and is more prone to bias.
The randomized controlled trial (RCT) is the golden standard for generating unbiased information on the benefits of an intervention. However, performing RCTs in most rare renal diseases is a most challenging undertaking. These conditions affect a few thousand or, sometimes, even fewer than one hundred patients on a world scale and therefore trials are often unrealistic. A RCT of sufficient power would need to recruit patients from very large areas over long periods, so that financial constraints are prohibitory for setting up this sort of trials. National or international registries, medical information platforms and regularly updated portals (see Table 2) may be useful to increase awareness and improve knowledge of rare conditions and to maximize recruitment of patients from multiple centres/countries thereby avoiding fragmentation of data collection. Furthermore, some rare renal diseases have also early manifestations in childhood. Although the need for RCTs in children is increasingly recognized in paediatric research and legislative measures have been taken by local governments to encourage RCTs in children, the enrollment of a large number of paediatric patients remains problematic. Finally, some rare kidney diseases are characterized by a very long course with variable phenotypic manifestation, making the follow-up and the correct definition of outcomes challenging.
HOW CAN THE QUALITY OF EVIDENCE BE ENHANCED IN RARE RENAL DISEASES?
From a statistical point of view, the number of subjects recruited in a trial should be sufficient to reject the null hypothesis (no difference in effect between the interventions) in favour of the alternative hypothesis (a difference in effect exists between the interventions) for a given level of significance. The probability of reaching a true positive conclusion is known as the study power, which can be maximized by increasing the number of participants . Since enrollment of a large number of patients in rare diseases is challenging, alternative strategies to show a particular effect, if in reality there is one, have been proposed . Some of these techniques can be considered ‘neutral’, such as (i) use of a crossover design (ii) performing repeated measurements and (iii) analysis of co-variance instead of single comparison of outcome between groups. Others are more dubious, and do not always lead to more or better evidence, such as use of composite or surrogate outcomes and interim analyses. Other ‘tailored’ designs have also been proposed to reduce the sample needed to reach statistical significance. In ‘sequential’ designs, for instance repeated statistical analyses are performed on accumulating data, and the study is stopped as soon as the information is sufficient to reach conclusions . Study designs exist that do not contemplate a pre-study calculation of sample size. In this ‘adaptive’ design approach, modifications after interim analyses are permitted. For example, on the basis of observed results at a certain time point, the required sample size for further recruitment can be re-assessed . Such an adaptive design was adopted in the PRIMO (Paricalcitol capsules benefits in Renal failure-Induced cardiac MOrbidity) study . In this study, because of limited prior data to estimate sample size, an interim efficacy analysis at a pre-specified time point and conditioned on the status of enrollment was planned for eventual sample size re-estimation. The decision to increase sample size was based on the observed treatment effect of paricalcitol on left ventricular hypertrophy. However, all these ‘methodological tips and tricks’ are always a slippery slope, and one should always be very critical in their interpretation.
In ‘open-ended’ RCTs patients can be continuously enrolled until a reliable positive or negative conclusion about the treatments is made. The single-patient (n-of-1) clinical trial is another interesting opportunity in at least some diseases. This trial design is a multiple crossover study conducted in single individuals where a single enrolled patient is exposed repeatedly to different treatments in a randomized order. Unlike traditional trials, the unit of randomization is the intervention rather than the patient, and the outcome is the conclusion about the best treatment for a particular patient . Hackett et al. conducted a study with this design on a patient affected by ornithine transcarbamylase (OTC) carrier deficiency, a rare X-linked disorder of the urea cycle with variable clinical manifestations in females, including end-stage renal disease. The treating physician and patient were blinded to treatment (l-arginine, an obligate OTC carrier). Either placebo capsules or l-arginine capsules were given for weekly periods. Plasma arginine and glutamine levels and changes in quality of life were considered as weekly efficacy indicators. After the end of the study a clear benefit of l-arginine compared with placebo was demonstrated for that particular patient. Other acceptable statistical approaches to trials in small populations include the adoption of less conservative statistical cut-off levels, such as P-values >5% and the use of one-side instead of two-sides tests of significance . Although one-sided tests are commonly considered not trustworthy, these types of tests can be used to maximize the power of effect detection if one knows exactly which is the direction of the effect that has to be expected. For example, let us imagine we want to test a new drug able to prevent renal stones formation in patients affected by cystinosis. The drug is cheaper than the other available drugs and previous evidence shows that it was efficacious to prevent lithiasis in all cases. Under these circumstances, given that we exactly know the direction of the effect (only improvement and no worsening), it would be appropriate to use a one-sided test, i.e. a test requiring a smaller sample size when compared with a two-sided traditional test. Less conservative approaches can be applied also to the assessment of the study power, starting from the assumption that confidence intervals (CIs) of treatment effect estimation, although closely linked to the statistical significance, are much more informative than P-values. Generally, for a given treatment effect, the confidence interval (CI) is reduced as the sample size is increased. However, when the recruitment of a high number of patients is difficult, such as in rare diseases, the sample size could be reduced at the cost of a reasonable reduction in the precision of effect estimation (see Figure 1).
CAN WE PROVIDE ACCEPTABLE GUIDANCE IN RARE RENAL DISEASES?
Clinical practice guidelines are built up on the basis of evidence-based medicine but available scientific knowledge and available good quality studies are ranked according to an accepted ‘evidence ladder’ for studies related to intended effects of therapy:
Systematic reviews of reliable RCTs including meta-analyses with low-heterogeneity between the results.
Other meta-analyses of RCTs.
Meta-analyses of observational studies (cohorts and case–control).
Single observational studies (cohorts and case–control).
Published case series/case-reports.
In rare renal diseases, the basis of evidence for treatments is generally weak and, in very rare diseases, the combined evaluation of single-case studies is often the only evidence that can be obtained. Can principles of clinical guidance applied to common diseases be appropriate also for rare diseases? A systematic search by the German Institute for Quality and Efficiency in Health Care (IQWiG) revealed no different methodological approaches for the development of clinical practice recommendations in rare diseases when compared with guidelines for common disorders . This document underlined that the issue of ‘scarce evidence’ is a limitation pertaining also guidelines on common diseases. Therefore, in principle, there is no reason to adopt a different approach or pose different requirements for rare diseases. A position document by the European Medicines Agency (EMEA)  supports the concept that rules for the production of guidelines on common diseases are also applicable to rare diseases. EMEA emphasizes that all forms of evidence, even anecdotal case-reports, may provide relevant information and should then be taken into consideration.
THE IMPORTANCE OF INTERNATIONAL REGISTRIES
In this scenario establishing large, international registries is of paramount importance. These registries may indeed help in the assessment of effectiveness and safety of treatments and even represent an important source for historical controls. The International registry of Hemolytic Uremic Syndrome (HUS) and Thrombotic Thrombocytopenic Purpura (TTP) is a successful example [14, 15]. The registry has been established with the aim to study the genetic and biochemical abnormalities of HUS/TTP, collect clinical and genetic data of patients and their families, find the best therapeutic approach for patients and provide up-to-date information to physicians and families. The registry has been established in 1996 and until now 730 cases of HUS/TTP have been referred to the registry from about 180 European and extra-European centres. The International Registry for Hereditary Kidney Stone Diseases is an ongoing initiative started in July 2003  which collects medical information worldwide from over 1100 patients affected by Primary Hyperoxaluria (PH), Dent disease, Cystinuria and Adenine phosphoribosyltransferase (APRT) deficiency with the aim to increase knowledge about these rare disorders, provide evidence to establish patient care guidelines and create the basis for future clinical trials. Clinical and laboratory data from 203 subjects have been recorded so far. Other international registers on rare renal diseases are the UK Registry for Rare Kidney Diseases (RADAR)  and the Cure Cystinosis International Registry (CCIR) .
PLANNING INTERVENTIONAL STUDIES IN RARE DISEASES
When designing a drug trial in a rare disease, a detailed background on the pharmacology of a drug being tested is desirable and pharmacology studies can also help to identify sources of heterogeneity in patients affected by the disease. For rare diseases with very long course, the use of surrogate end points may be acceptable but the relationship between these surrogates and clinical efficacy should be preliminarily established to properly evaluate the balance of risks and benefits.
An example of validated surrogate end point is cyst volume in patients with ADPKD. Kidney enlargement resulting from the expansion of cysts is a quantifiable parameter and higher rates of kidney enlargement reflect a more rapid decrease in renal function .
No specific and validated methods for designing, carrying out or analysing clinical trials in small populations exist. Although high-quality data adhering to good clinical practice standards are desirable, a more pragmatic attitude can be justified in certain situations. However, a careful balance between potential benefit and harm, not only for the individual patient but also for the society at large, should be maintained, to avoid either that effective treatments are withheld, or useless treatments provided. At least, the guidance should be made transparent, by providing a crisp description of how the literature was searched, how retrieved evidence was interpreted, and what considerations were taken into account when formulating the recommendation. The use of ‘expert groups’ supported by a team of methodologists, where consensus between experts in the field of the disease is obtained during balanced face-to face discussions, might be a possible way to balance personal experience and available (as few as it might be) evidence.
- © The Author 2013. Published by Oxford University Press on behalf of ERA-EDTA. All rights reserved.