Weekly Data Processing

Table of Contents

  1. Weekly Data Processing
    1. Qualtrics Data
    2. MindLAMP Data
    3. MRI Data
    4. Task Data
    5. Check all data from PCX to Flywheel
    6. Do McLean Scan Download Process

Qualtrics Data

Download qualtrics data to amarel using the Qualtrics API

  1. Log into amarel
  2. Navigate to /projects/f_ah1491_1/analysis_tools/qualtrics/download_qualtrics.ipynb
  3. Run the notebook
  4. Check the data has appeared in /scratch/f_ah1491_1/internal_data/PCX/behavioral

Download qualtrics data to Box using the Qualtrics API

  1. Navigate to /Users/demo/Library/CloudStorage/Box-Box/Holmes_Lab_Wiki/PCX_Round2/Data_processing/download_qualtrics.ipynb
  2. Run the notebook
  3. Check the data has appeared in ~/Library/CloudStorage/Box-Box/(Restricted)_PCR/PCX/behavioral

MindLAMP Data

  1. Connect to MGB VPN (to access HPC Eris)
  2. Run
rsync -avz kj110@eristwo.partners.org:'/data/sbdp/PHOENIX/GENERAL/PCX/PC*' '/Users/demo/Library/CloudStorage/Box-Box/(Restricted)_PCR/PCX/mindlamp_mri_data/data'
  1. Connect to Rutgers VPN (to access HPC Amarel)
  2. Run
rsync -avz '/Users/demo/Library/CloudStorage/Box-Box/(Restricted)_PCR/PCX/mindlamp_mri_data/data' kj537@amarel.rutgers.edu:'/scratch/f_ah1491_1/internal_data/PCX'

MRI Data

  1. Log into flywheel using this tutorial: https://holmeslab.github.io/holmeslab/docs/Experiments/flywheel-tutorial/
  2. Check each recent person has
    • All relevant scans
    • Echos are in the appropriate scan container if the scan was repeated
  3. Process data from Raw to BIDS using this tutorial:https://holmeslab.github.io/holmeslab/docs/Experiments/flywheel-bids/
  4. Process BIDS data to fMRIPrep using this tutorial: https://holmeslab.github.io/holmeslab/docs/Experiments/flywheel-fmriprep/
  5. Download BIDS and fMRIPrep data using this tutorial: https://holmeslab.github.io/holmeslab/docs/PCX/download-mri/

Task Data

When you run tasks from files in Box (through Psychopy), the data saves to the same Box location in the /data folder. Send task data from Box to Amarel in BIDS format by running this script:

~/Library/CloudStorage/Box-Box/Holmes_Lab_Wiki/PCX_Round2/Data_processing/create_bids_files.ipynb

Check all data from PCX to Flywheel

Log into Flywheel and make sure all the data from the past week at Rutgers is there.

Do McLean Scan Download Process

Overview of Data Pipeline