Name: meta-analysis of voxel-based neuroimaging studies using seed-based d mapping with permutation of subject images (sdm-psi)
Uploaded: 2019-11-27T12:30:00
Duration: 6 min 26 s
Description: Watch this Scientific Journal Video about Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI at JoVE.com

This work was supported by Miguel Servet Research Contract MS14/00041 and Research Project PI14/00292 from the Plan Nacional de I+D+i 2013&#8211;2016, the Instituto de Salud Carlos III-Subdirecci&#243;n General de Evaluaci&#243;n y Fomento de la Investigaci&#243;n, the European Regional Development Fund (FEDER), and by PFIS Predoctoral Contract FI16/00311. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

1. Installation of SDM-PSI
<ol>
	<li>Go to <a href="https://www.sdmproject.com/software/" target="_blank">https://www.sdmproject.com/software/</a> to download the version of SDM-PSI for the operating system of the computer as a ZIP file.</li>
	<li>Decompress the ZIP file. To avoid problems, decompress it within a local folder without blank spaces in its path.</li>
	<li>Click the file SdmPsiGui to execute the graphical interface of SDM-PSI, and close the About splash window that will automatically open.</li>
	<li>If SdmPsiGui does not find all required paths, it will automatically offer to display the preferences window. Press Yes.
	<ol>
		<li>If MRICron is not installed in the computer go to <a href="https://www.nitrc.org/frs/?group_id=152" target="_blank">https://www.nitrc.org/frs/?group_id=152</a> to download the version for the operating system of the computer as a ZIP file, and decompress the ZIP file.</li>
		<li>In the Brain viewer tab of the preferences window, ensure that Brain viewer is set to MRICron, and click the folder icon next to Brain viewer executable to find the MRICron executable.</li>
		<li>Ensure that all remaining paths in the different tabs have blue marks, which indicate that the paths are correct.</li>
	</ol>
	</li>
	<li>In case that SdmPsiGui has not automatically displayed the preferences window, go to the Tools menu and click Preferences.</li>
	<li>In the Multithreading tab, specify the number of concurrent threads to use in the calculations. Some SDM-PSI calculations take a very long time (from hours to days) and consume a large amount of RAM memory (from hundreds of megabytes to gigabytes). The use of multiple threads (parallel processing) substantially decreases the time but increases the memory used.</li>
	<li>Close the preferences window and SdmPsiGui.</li>
</ol>
2. Meta-analysis plan
<ol>
	<li>Specify a precise question. For instance, &#34;do patients with OCD have gray matter volume regional abnormalities?&#34;</li>
	<li>Write clear inclusion criteria that allow a systematic inclusion of the studies. For instance, &#34;all studies that performed whole-brain voxel-based comparison of gray matter volume between individuals with OCD and healthy controls&#34;.</li>
	<li>Write clear exclusion criteria that allow a systematic exclusion of those studies that cannot or should not be included for specific reasons. For instance, &#34;studies with less than 10 patients, duplicated datasets, or studies from which the required information cannot be retrieved&#34;.</li>
	<li>Write down the data to extract from each study. The following list includes the recommended data (some of them are not strictly required, but their absence would impoverish the meta-analysis):
	<ul>
		<li>An identification of the study.</li>
		<li>The sample sizes.</li>
		<li>The statistical significance level, which is the t-value, z-value or p-value used in the study to determine which voxels were statistically significant.</li>
		<li>The software and stereotactic space. See Table 1 for the list of software packages and stereotactic spaces understood by SDM-PSI.</li>
		<li>The coordinates and height of the peaks. The height of a peak is its t-value or z-value, but a p-value is also useful.</li>
		<li>Variables that will be used to describe the samples or to conduct subgroup analyses or meta-regressions.</li>
	</ul>
	</li>
	<li>To increase the quality of the review, consider following the &#34;Ten simple rules for neuroimaging meta-analysis&#34;20 and the PRISMA checklist21.</li>
	<li>To increase the transparency of the review, consider registering the protocol beforehand on a publicly available database such as PROSPERO (<a href="https://www.crd.york.ac.uk/PROSPERO/" target="_blank">https://www.crd.york.ac.uk/PROSPERO/</a>).</li>
</ol>
3. Exhaustive search
<ol>
	<li>Select a set of keywords that allow finding any study that might meet the inclusion criteria. For instance, the keywords might be &#34;obsessive&#8211;compulsive disorder&#34; plus &#34;morphometry&#34;, &#34;voxel-based&#34; or &#34;voxelwise&#34;.</li>
	<li>Conduct the search on databases such as PubMed and Web of Science:
	<ol>
		<li>Go to database website, e.g. <a href="https://www.ncbi.nlm.nih.gov/pubmed/" target="_blank">https://www.ncbi.nlm.nih.gov/pubmed/</a> for PubMed.</li>
		<li>Type the search query. In the example meta-analysis, the query might be &#34;obsessive&#8211;compulsive disorder&#34; AND (&#34;morphometry&#34; OR &#34;voxel-based&#34; OR &#34;voxelwise&#34;). In this query, the operator &#34;AND&#34; means that the studies must have all keywords, the operator &#34;OR&#34; means that the studies must have at least one of the keywords, and the parentheses indicate the order of these logical operations. Therefore, the retrieved studies must have the keyword &#34;obsessive&#8211;compulsive disorder&#34; and at least one of the keywords &#34;morphometry&#34;, &#34;voxel-based&#34; or &#34;voxelwise&#34;. Note that other strategies are possible.</li>
	</ol>
	</li>
	<li>Apply the inclusion/exclusion criteria. For instance, from the results provided by the databases, select only the articles that analyze differences in gray matter volume between patients with obsessive-compulsive disorder and controls performing whole-brain voxel-based morphometry studies, and discard studies including less than 10 patients and studies that re-analyzed previously published data.</li>
	<li>To increase the exhaustiveness of the search, consider conducting a manual search over the referenced works in the selected studies.</li>
	<li>To maximize the inclusion of studies and avoid uncertainty in the collection of data, consider contacting the corresponding authors to ask for any missing or unclear data.</li>
	<li>Record the number of studies retrieved and the number of studies excluded for each reason. Consider creating a PRISMA flow diagram21 with these numbers.</li>
</ol>
4. Collection of the data
<ol>
	<li>For each included study, read the manuscript to find the specific data to extract.</li>
	<li>Save the data from the studies systematically, e.g., typing the data in pre-formatted spreadsheet files. To minimize typing errors, consider copying and pasting the numbers of interest and double-checking the saved data.</li>
	<li>When the statistical significance level is unclear, consider following these recommendations:
	<ol>
		<li>If the manuscript reports peaks obtained using two whole-brain statistical significance levels, e.g. p-value &#60; 0.001 without correction for multiple comparisons (from now on, &#34;uncorrected threshold&#34;) and familywise error rate (FWER) &#60; 0.05 (from now on &#34;corrected threshold&#34;), select the uncorrected threshold, and include all peaks obtained using this uncorrected threshold. The reason to prefer the uncorrected threshold is that studies usually obtain more peaks applying an uncorrected threshold, and SDM estimates the maps more accurately if it has information from more peaks.</li>
		<li>If the manuscript reports peaks obtained using an uncorrected threshold for increases and a corrected threshold for decreases (or vice versa), select the uncorrected threshold but include only the peaks obtained using the corrected threshold. This is a conservative approximation because it might discard some peaks obtained using the uncorrected threshold. An example of this situation might be when a manuscript states something such as &#34;we detected FWER-corrected larger gray matter volumes in some regions, while we did not detect smaller gray matter volume in any region even using uncorrected p-value &#60; 0.001&#34;.</li>
		<li>If the authors applied cluster-based statistics, use the cluster-forming height threshold. This is a conservative approximation because some voxels might have had t-values higher than the threshold, but the authors discarded them because their clusters were not large enough.</li>
		<li>If the manuscript does not specify a threshold, use a slightly smaller value than the t-value of the smallest peak. The reason to use this t-value is that if the authors had applied this statistical significance threshold without requiring a minimum size for the clusters, they would have found the same peaks.</li>
	</ol>
	</li>
	<li>When recording the peak information do the following:
	<ol>
		<li>Exclude peaks obtained using a statistical significance threshold that is more liberal than the threshold selected for the rest of the brain. An example of this situation is when the authors applied more liberal thresholds or small volume corrections to some a priori brain regions.</li>
		<li>Convert z-values and p-values into t-values. Click the button Convert peaks in the SDM-PSI software to convert them easily. Alternatively, convert them in the same spreadsheet file (e.g., &#34;=T.INV(1-0.001,34)&#34; for p-value = 0.001 and 34 degrees of freedom; the degrees of freedom are the sum of the sample sizes minus the number of parameters, which in a two-sample comparison are two plus the number of covariates used in the original comparison).</li>
		<li>Use positive t-values for peaks of increase (e.g., activation) and negative t-values for peaks of decrease (e.g., deactivation). See Table 2 for guidance on how to decide the sign of the t-values. 
		NOTE: We obtained information from studies &#34;Heuvel&#34; and &#34;Soriano-Mas&#34; after personal communication.</li>
	</ol>
	</li>
</ol>
5. Introduction of data into SDM-PSI
<ol>
	<li>Open SdmPsiGui and close the About splash window (avoid having any key pressed while closing it).</li>
	<li>Click the Change meta-analysis button at the upper-left part of the graphical interface to select a directory for the meta-analysis (any new empty directory of choice will do).</li>
	<li>Click the button SDM table editor to input general information from the studies, including their identification (column &#34;study&#34;), their sample sizes (columns &#34;n1&#34; and &#34;n2&#34;), the t-value that they used as statistical thresholds (column &#34;t_thr&#34;), and other potential variables to conduct subgroup analyses or meta-regressions.</li>
	<li>Within the selected directory, create a text file for each study with the coordinates and t-value of each peak:
	<ol>
		<li>Open a text editor to create a text file named as [the identification of study] + &#34;.&#34; + [the software] + &#34;_&#34; + [the stereotactic space] + &#34;.txt&#34;. For instance, for the study &#34;Carmona&#34; that was conducted with SPM and reports the coordinates in MNI space, the name of the text file must be &#34;Carmona.spm_mni.txt&#34;. If the study has no peaks, the software and stereotactic space can be replaced by &#34;no_peaks&#34;.</li>
		<li>Write the coordinates and t-value of each peak in a different row. For instance, the first rows of the text file &#34;Carmona.spm_mni.txt&#34; should be: 
		40,39,21,-5.14 
		53,27,21,-3.77 
		56,23,20,-3.63</li>
	</ol>
	</li>
</ol>
6. Pre-processing
<ol>
	<li>Click in the Preprocessing button at the left menu bar, select the modality of the studies at the list box labeled &#34;Modality&#34; and press OK. In the example meta-analysis, the modality is &#34;VBM &#8211; gray matter&#34;.</li>
	<li>Wait (some minutes) while SDM-PSI calculates the maps of the lower and upper bounds of potential effect sizes. SdmPsiGui will show four progress bars that display the status of the execution and the expected remaining time for the current process. During the calculations, the color of the circle next to &#34;Processing status&#34; will be yellow, and will change to green if the execution ends successfully, or to red if it fails.</li>
</ol>
7. Main analysis
<ol>
	<li>Click the Mean button at the left menu bar and press OK.</li>
	<li>Wait (some minutes) while SDM-PSI conducts the multiple imputation and meta-analysis (Figure 1).</li>
	<li>Click the Threshold button in the left menu toolbox, select the uncorrected p-values of the main analysis (&#34;MyTest_uncorrp&#34; by default) and press OK. SDM-PSI will automatically open both MRICron to visualize the results and a webpage with a detailed report of them.</li>
	<li>Press the FWE correction button at the left menu toolbox, select the main analysis in the list box (&#34;MyTest&#34; by default) and press OK.</li>
	<li>Wait (some hours or even days) while SDM-PSI conducts the permutation test.</li>
	<li>Click the Threshold button in the left menu toolbox, select the TFCE-correction of the main analysis (&#34;MyTest_corrp_tfce&#34; by default) and press OK. SDM-PSI will automatically open both MRICron to visualize the results and a webpage with a detailed report of them.</li>
</ol>
8. Heterogeneity, publication bias and grading
<ol>
	<li>Click the Extract button on the left menu toolbox, select a peak from the main analysis (&#34;MyTest_z_p_0.05000_10_neg_peak1&#34; by default) and press OK. SDM-PSI will automatically open a webpage with statistics of this peak. Write down the heterogeneity I2 statistic.</li>
	<li>Click the Bias Test button on the left menu toolbox, select a peak from the main analysis (&#34;MyTest_z_p_0.05000_10_neg_peak1&#34; by default) and press OK. SDM-PSI will automatically open a webpage with a funnel plot and the results of a test for small-study effect and a test for excess significance. The former tests whether there is asymmetry in the funnel plot (i.e., larger effect size in small studies), which could indicate that small studies are only published if they find large effect sizes or other sources of bias. The latter tests whether the number of studies with statistically significant results is larger than expected, which could indicate that studies are only published if they find statistically significant results or other sources of bias.</li>
	<li>Press the Evidence grading button from the top toolbox, select the main analysis (&#34;MyTest&#34; by default) from the list box and press OK. After some seconds, SDM-PSI will automatically open MRICron to show the classes of evidence.</li>
</ol>

<ol>
	<li>Turkeltaub, P. E., Eden, G. F., Jones, K. M., Zeffiro, T. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Meta-analysis+of+the+functional+neuroanatomy+of+single-word+reading:+method+and+validation.">Meta-analysis of the functional neuroanatomy of single-word reading: method and validation.</a> Neuroimage. 16 (3 Pt 1), 765-780 (2002).</li><li>Laird, A. R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=ALE+meta-analysis:+controlling+the+false+discovery+rate+and+performing+statistical+contrasts.">ALE meta-analysis: controlling the false discovery rate and performing statistical contrasts.</a> Human Brain Mapping. 25 (1), 155-164 (2005).</li><li>Wager, T. D., Lindquist, M., Kaplan, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Meta-analysis+of+functional+neuroimaging+data:+current+and+future+directions.">Meta-analysis of functional neuroimaging data: current and future directions.</a> Social Cognitive and Affective Neuroscience. 2 (2), 150-158 (2007).</li><li>Radua, J., Mataix-Cols, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Voxel-wise+meta-analysis+of+grey+matter+changes+in+obsessive-compulsive+disorder.">Voxel-wise meta-analysis of grey matter changes in obsessive-compulsive disorder.</a> British Journal of Psychiatry. 195 (5), 393-402 (2009).</li><li>Eickhoff, S. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Coordinate-based+activation+likelihood+estimation+meta-analysis+of+neuroimaging+data:+a+random-effects+approach+based+on+empirical+estimates+of+spatial+uncertainty.">Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: a random-effects approach based on empirical estimates of spatial uncertainty.</a> Human Brain Mapping. 30 (9), 2907-2926 (2009).</li><li>Eickhoff, S. B., Bzdok, D., Laird, A. R., Kurth, F., Fox, P. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Activation+likelihood+estimation+meta-analysis+revisited.">Activation likelihood estimation meta-analysis revisited.</a> Neuroimage. 59 (3), 2349-2361 (2012).</li><li>Albajes-Eizagirre, A., Radua, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+do+results+from+coordinate-based+meta-analyses+tell+us?.">What do results from coordinate-based meta-analyses tell us?.</a> Neuroimage. 176, 550-553 (2018).</li><li>Albajes-Eizagirre, A., Solanes, A., Vieta, E., Radua, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Voxel-based+meta-analysis+via+permutation+of+subject+images+(PSI):+Theory+and+implementation+for+SDM.">Voxel-based meta-analysis via permutation of subject images (PSI): Theory and implementation for SDM.</a> Neuroimage. 186, 174-184 (2018).</li><li>Bossier, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Influence+of+Study-Level+Inference+Models+and+Study+Set+Size+on+Coordinate-Based+fMRI+Meta-Analyses.">The Influence of Study-Level Inference Models and Study Set Size on Coordinate-Based fMRI Meta-Analyses.</a> Frontiers in Neuroscience. 11, 745 (2017).</li><li>Radua, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+new+meta-analytic+method+for+neuroimaging+studies+that+combines+reported+peak+coordinates+and+statistical+parametric+maps.">A new meta-analytic method for neuroimaging studies that combines reported peak coordinates and statistical parametric maps.</a> European Psychiatry. 27 (8), 605-611 (2012).</li><li>Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M., Nichols, T. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Permutation+inference+for+the+general+linear+model.">Permutation inference for the general linear model.</a> Neuroimage. 92, 381-397 (2014).</li><li>Smith, S. M., Nichols, T. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Threshold-free+cluster+enhancement:+addressing+problems+of+smoothing,+threshold+dependence+and+localisation+in+cluster+inference.">Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference.</a> Neuroimage. 44 (1), 83-98 (2009).</li><li>Radua, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Anisotropic+kernels+for+coordinate-based+meta-analyses+of+neuroimaging+studies.">Anisotropic kernels for coordinate-based meta-analyses of neuroimaging studies.</a> Frontiers in Psychiatry. 5, 13 (2014).</li><li>Albajes-Eizagirre, A., Solanes, A., Radua, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Meta-analysis+of+non-statistically+significant+unreported+effects.">Meta-analysis of non-statistically significant unreported effects.</a> Statistical Methods in Medical Research. 962280218811349, (2018).</li><li>Durnez, J., Moerkerke, B., Nichols, T. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Post-hoc+power+estimation+for+topological+inference+in+fMRI.">Post-hoc power estimation for topological inference in fMRI.</a> Neuroimage. 84, 45-64 (2014).</li><li>Fullana, M. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fear+extinction+in+the+human+brain:+A+meta-analysis+of+fMRI+studies+in+healthy+participants.">Fear extinction in the human brain: A meta-analysis of fMRI studies in healthy participants.</a> Neuroscience &amp; Biobehavioral Reviews. 88, 16-25 (2018).</li><li>Wise, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Common+and+distinct+patterns+of+grey-matter+volume+alteration+in+major+depression+and+bipolar+disorder:+evidence+from+voxel-based+meta-analysis.">Common and distinct patterns of grey-matter volume alteration in major depression and bipolar disorder: evidence from voxel-based meta-analysis.</a> Molecular Psychiatry. 22 (10), 1455-1463 (2017).</li><li>Radua, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multimodal+voxel-based+meta-analysis+of+white+matter+abnormalities+in+obsessive-compulsive+disorder.">Multimodal voxel-based meta-analysis of white matter abnormalities in obsessive-compulsive disorder.</a> Neuropsychopharmacology. 39 (7), 1547-1557 (2014).</li><li>He, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Meta-analytic+comparison+between+PIB-PET+and+FDG-PET+results+in+Alzheimer's+disease+and+MCI.">Meta-analytic comparison between PIB-PET and FDG-PET results in Alzheimer's disease and MCI.</a> Cell Biochemistry and Biophysics. 71 (1), 17-26 (2015).</li><li>Muller, V. I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ten+simple+rules+for+neuroimaging+meta-analysis.">Ten simple rules for neuroimaging meta-analysis.</a> Neuroscience &amp; Biobehavioral Reviews. 84, 151-161 (2018).</li><li>Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Group, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Preferred+reporting+items+for+systematic+reviews+and+meta-analyses:+the+PRISMA+statement.">Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.</a> British Medical Journal. 339, b2535 (2009).</li><li>Radua, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+causes+psychosis?+An+umbrella+review+of+risk+and+protective+factors.">What causes psychosis? An umbrella review of risk and protective factors.</a> World Psychiatry. 17, 49-66 (2018).</li></ol>

The authors have nothing to disclosure.

As introduced earlier, most voxel-based meta-analytic methods use a test for convergence of peaks that has some limitations, and then conduct a binary classification of the evidence exclusively based on p-values.
In this protocol, we detailed how to conduct a voxel-based meta-analysis using SDM-PSI, which has a number of positive features including a standard permutation test to assess the statistical significance of the effects. In addition, we show how the strength of the evidence can be graded using a set of criteria that go beyond a binary classification that solely relies on one statistical significance level.
To facilitate the replication of the example meta-analysis, we provide the data already extracted from the manuscripts from a previous meta-analysis. Interestingly, in the manuscript of that meta-analysis, the evidence &ldquo;seemed&rdquo; stronger than the evidence that we found with the updated methods. We therefore suggest that unsystematic evaluations of the evidence in previous voxel-based meta-analyses are taken with caution.
We hope that following this protocol, neuroimaging meta-analyses provide a richer and more granulate description of the evidence of neuroimaging findings.

Since the introduction of magnetic resonance imaging, the neuroimaging community has published thousands of studies of the neural substrates of psychological functions and neuropsychiatric disorders. To summarize these findings, several methods have been developed1,2,3,4,5,6. Original voxel-based neuroimaging studies report the coordinates of the peaks of statistical significance (e.g., in a comparison of gray matter volume between patients and controls), and meta-analytic methods commonly assess whether there is convergence of peaks in certain brain regions.
However, we have previously shown that these tests for convergence of peaks rely on delicate assumptions that might influence the patterns of meta-analysis results and their statistical significance7. Specifically, these tests assume that voxels are independent and that they have the same probability of a "false" peak, while in real gray matter, voxels correlate with their neighbors and the probability that a voxel has a "false" peak depends on its tissue composition. In addition, they also encompass paradoxes such as that the statistical power increases in the presence of few true effects, and decreases when there are multiple true effects.
To overcome these problems, we developed a method that imputes the brain maps of statistical effects for each study and then conducts a standard random-effects meta-analysis to formally test whether the effects are different from zero. This method is called "Seed-based d Mapping with Permutation of Subject Images" (SDM-PSI)8 and its main features include:
<ul>
	<li>Accounting for both increases and decreases of the outcome of interest (e.g., activation and deactivation) so that contradictory findings cancel each other4;</li>
	<li>Use of effect size estimates with random-effects modeling, which increases reliability and performance9;</li>
	<li>Potential simultaneous inclusion of available 3D statistical images (i.e., maps of t-test values)10;</li>
	<li>Subject-based permutation tests identical to those of FSL "randomize" tool11;</li>
	<li>Use of threshold-free cluster enhancement (TFCE) statistics12.</li>
</ul>
We have detailed and fully validated the SDM methods elsewhere4,8,10,13,14.
In addition, we suggest not relying on a binary classification of the voxels based on the level of statistical significance (significant vs. not significant) but, conversely, assessing the strength of the evidence using a set of criteria22. The binary statistical significance reductionism leads to poor control of the false positive and false negative rates15, whereas the criteria use ranges of statistical significance levels and take into account the amount of data or potential biases. The SDM-PSI software returns the necessary elements to conduct such a classification8 and thus they can be employed to afford a more granular classification of the strength of the evidence.
Here we show how to conduct a meta-analysis of voxel-based neuroimaging studies using SDM-PSI. To exemplify the protocol, we use data from a published meta-analysis of voxel-based morphometry studies that investigated gray matter abnormalities in patients with obsessive-compulsive disorder (OCD)4. However, we will not use the methods employed in that early meta-analysis, but the aforementioned state-of-the-art procedures. The reader can download the software and these data from our website (<a href="http://www.sdmproject.com/" target="_blank">http://www.sdmproject.com/</a>) to replicate the analysis.
All researchers who aim to conduct a meta-analysis of voxel-based neuroimaging studies can follow this protocol. The method can be used with functional magnetic resonance imaging (fMRI, e.g., BOLD response to a stimulus)16, voxel-based morphometry (VBM, e.g., gray matter volume)17, diffusion tensor imaging (DTI, e.g., fractional anisotropy)18, position emission tomography (PET, e.g., receptor occupancy)19 and surface-based morphometry (SBM, e.g. cortical thickness) studies/datasets.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Computer</td>
			<td></td>
			<td></td>
			<td>32- or 64-bit Windows, 64-bit Mac OSX, or 64-bit Linux</td>
		</tr>
	</tbody>
</table>

meta-analysis of voxel-based neuroimaging studies using seed-based d mapping with permutation of subject images (sdm-psi)

Most methods for conducting meta-analysis of voxel-based neuroimaging studies do not assess whether effects are not null, but whether there is a convergence of peaks of statistical significance, and reduce the assessment of the evidence to a binary classification exclusively based on p-values (i.e., voxels can only be &ldquo;statistically significant&rdquo; or &ldquo;non-statistically significant&rdquo;). Here, we detail how to conduct a meta-analysis using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI), a novel method that uses a standard permutation test to assess whether effects are not null. We also show how to grade the strength of the evidence according to a set of criteria that considers a range of statistical significance levels (from more liberal to more conservative), the amount of data or the detection of potential biases (e.g., small-study effect and excess of significance). To exemplify the procedure, we detail the conduction of a meta-analysis of voxel-based morphometry studies in obsessive-compulsive disorder, and we provide all the data already extracted from the manuscripts to allow the reader to replicate the meta-analysis easily. SDM-PSI can also be used for meta-analyses of functional magnetic resonance imaging, diffusion tensor imaging, position emission tomography and surface-based morphometry studies.

As shown in the map opened in MRICron when thresholding of the main analysis (Figure 2, step 7.6), patients with OCD had statistically significantly smaller gray matter volume in dorsal anterior cingulate/medial frontal cortex. The accompanying webpage details that the cluster is moderately small (143 voxels) and mainly located at Brodmann area 32, and that the peak of the cluster is at MNI [2, 32, 32], has a z-value of -4.97 and a FWER-corrected p-value of 0.01.
In the webpages obtained in steps 8.1 and 8.2, the low I2 statistic (1.5%) indicates very small heterogeneity, the funnel plot does not show asymmetries (Figure 3), and both the test for small-study effect and the test for excess of significance are negative. However, the evidence of smaller gray matter volume in dorsal anterior cingulate cortex is moderately weak, as shown in the map open in MRICron when grading evidence, especially due to limited amount of data.
When the main analysis was thresholded using a less stringent statistical significance level (step 7.3), patients also showed statistically significantly larger gray matter volume in the striatum and superior parietal gyrus (uncorrected p-values = 0.00006 and 0.0002 respectively), but the evidence of these abnormalities should be considered weaker.

<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/59841/59841fig01.jpg" /> 
Figure 1: Main window of the SDP-PSI graphical user interface during a mean execution. <a href="https://www.jove.com/files/ftp_upload/59841/59841fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/59841/59841fig02.jpg" /> 
Figure 2: Regions of statistically significantly smaller gray matter volume in patients with obsessive-compulsive disorder as compared to matched healthy controls. 
The cluster of statistical significance covers 143 voxels, has its peak at MNI [2,32,32], and includes mostly dorsal anterior cingulate/medial frontal cortex, Brodmann area 32. <a href="https://www.jove.com/files/ftp_upload/59841/59841fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/59841/59841fig03.jpg" /> 
Figure 3: Funnel plot for the peak of the cluster of statistically significantly smaller gray matter volume in dorsal anterior cingulate cortex. <a href="https://www.jove.com/files/ftp_upload/59841/59841fig03large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Software packages</td>
			<td>Coding in SDM</td>
		</tr>
		<tr>
			<td>Statistical Parametric Mapping (SPM)</td>
			<td>spm</td>
		</tr>
		<tr>
			<td>FMRIB Software Library (FSL)</td>
			<td>fsl</td>
		</tr>
		<tr>
			<td>Other packages</td>
			<td>other</td>
		</tr>
		<tr>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Stereotactic space</td>
			<td>Coding in SDM</td>
		</tr>
		<tr>
			<td>Montreal Neurological Institute (MNI)</td>
			<td>mni</td>
		</tr>
		<tr>
			<td>Raw Talairach</td>
			<td>tal</td>
		</tr>
		<tr>
			<td>MNI converted to Talairach using Brett transform</td>
			<td>brett</td>
		</tr>
	</tbody>
</table>
Table 1: List of software packages and stereotactic spaces understood by SDM-PSI.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td></td>
			<td>t-values must be positive when:</td>
			<td>t-values must be negative when:</td>
		</tr>
		<tr>
			<td>One-sample fMRI studies</td>
			<td>task &#62; baseline (activation)</td>
			<td>task &#60; baseline (deactivation)</td>
		</tr>
		<tr>
			<td>Two-sample fMRI studies</td>
			<td>patients &#62; controls in task &#62; baseline (hyper-activation)</td>
			<td>patients &#60; controls in task &#62; baseline (hypo-activation)</td>
		</tr>
		<tr>
			<td></td>
			<td>patients &#60; controls in task &#60; baseline (failure of deactivation)</td>
			<td>patients &#62; controls in task &#60; baseline (hyper-deactivation)</td>
		</tr>
		<tr>
			<td>Two-sample VBM / FA studies</td>
			<td>patients &#62; controls (larger volume / FA)</td>
			<td>patients &#60; controls (smaller volume / FA)</td>
		</tr>
	</tbody>
</table>
Table 2: Sign of the t-values of the peaks.

Watch this Scientific Journal Video about Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI at JoVE.com

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

We detail how to conduct a meta-analysis of voxel-based neuroimaging studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI).

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI)

Most methods for conducting meta-analysis of voxel-based neuroimaging studies do not assess whether effects are not null, but whether there is a ...

Research

JoVE Journal

Neuroscience

Functional Mapping with Simultaneous MEG and EEG

We use magnetoencephalography (MEG) and electroencephalography (EEG) to locate and determine the temporal evolution in brain areas involved in the ...

Basics of Multivariate Analysis in Neuroimaging Data

Multivariate analysis techniques for neuroimaging data have recently received increasing attention as they have many attractive features that cannot be ...

Mapping Inhibitory Neuronal Circuits by Laser Scanning Photostimulation

Inhibitory neurons are crucial to cortical function. They comprise about 20% of the entire cortical neuronal population and can be further subdivided into ...

Functional Neuroimaging Using Ultrasonic Blood-brain Barrier Disruption and Manganese-enhanced MRI

Although mice are the dominant model system for studying the genetic and molecular underpinnings of neuroscience, functional neuroimaging in mice remains ...

3-D Imaging and Analysis of Neurons Infected In Vivo with Toxoplasma gondii

3-D Imaging and Analysis of Neurons Infected In Vivo with Toxoplasma gondii

Toxoplasma gondii is an obligate, intracellular parasite with a broad host range, including humans and rodents. In both humans and rodents, Toxoplasma ...

Preparation of Acute Brain Slices Using an Optimized N-Methyl-D-glucamine Protective Recovery Method

Preparation of Acute Brain Slices Using an Optimized N-Methyl-D-glucamine Protective Recovery Method

This protocol is a practical guide to the N-methyl-D-glucamine (NMDG) protective recovery method of brain slice preparation. Numerous recent studies have ...

The Use of Trace Eyeblink Classical Conditioning to Assess Hippocampal Dysfunction in a Rat Model of Fetal Alcohol Spectrum Disorders

Neonatal rats were administered a relatively high concentration of ethyl alcohol (11.9% v/v) during postnatal days 4-9, a time when the fetal brain ...

Live Images of GLUT4 Protein Trafficking in Mouse Primary Hypothalamic Neurons Using Deconvolution Microscopy

Type 2 diabetes mellitus (T2DM) is a global health crisis which is characterized by insulin signaling impairment and chronic inflammation in peripheral ...

Neuroimaging Field Methods Using Functional Near Infrared Spectroscopy (NIRS) Neuroimaging to Study Global Child Development: Rural Sub-Saharan Africa

Neuroimaging Field Methods Using Functional Near Infrared Spectroscopy NIRS Neuroimaging to Study Global Child Development: Rural Sub-Saharan Africa

Portable neuroimaging approaches provide new advances to the study of brain function and brain development with previously inaccessible populations and in ...

Ablation of a Neuronal Population Using a Two-photon Laser and Its Assessment Using Calcium Imaging and Behavioral Recording in Zebrafish Larvae

To identify the role of a subpopulation of neurons in behavior, it is essential to test the consequences of blocking its activity in living animals. Laser ...