Let’s role play

What most students do…

• They create a “intership folder” and put everything in it, from intership contract, pdf of papers and code

Let’s get organized

• You are likely to be doing many things during your thesis/intership:
  • litterature report
  • write a report or a thesis
  • do analyses for your first chapter…
  • for your second chapter…

Step 1: create a folder to store each analysis

mkdir ~/sandbox/myinternship

Step 1: create a folder to store each analysis

In this folder, you will organize your work in projects or subprojects.
Your first project is this training session.

mkdir ~/sandbox/myinternship/initiation_git
mkdir ~/sandbox/myinternship/analyse_chapitre1
mkdir ~/sandbox/myinternship/rapport

Step 2: Organize your working directory for the project

cd ~/sandbox/myinternship/initiation_git/
mkdir data
mkdir results
mkdir figures
mkdir scripts

mkdir "don't do that! Pitié@&!"

mkdir possible_long_folder_name
mkdir AlsoPossibleName

Step 3: Use a flavor of Markdown to document your work

Rmd, md, Qmd are text formats that allow combining analyses and text, and therefore to:

Step 3: Use a flavor of Markdown to document your work

• If you use R: Rstudio is recommended https://posit.co/downloads/
• If else, if you need to work with multiple languages, or if you need to work on a distant server: Visual Studio Code https://code.visualstudio.com

Create files in the directory

• Open Rstudio
• create a new project in an existing folder
• choose the folder we’ve just created
• create
• New file > Rmd
• Save the file under the name you want.

Why git?

• Imagine you are working on script (or a Rmd file)
• obviously, you’re not going to write the code in one go and be satisfied
• write, test, modify, start again…

Why git?

flowchart LR
  A[Data input] --> B(script1)
  B --> C(output1)
  C --> D(script2.V1)
  C --> D2(script2.V2)
  C --> D3(script2.V3)
  D --> E[Results]
  D2 --> E[Results]
  D3 --> E[Results]

What is git?

• Git is a version control sofware
• keeps the history of all the modifications made to all the files in a repository
• allows going back to an earlier version of the project, or anyfile…
• allows comparing versions
• and create branches if you want to “try something” ¹

Why this git training?

• Lots of ressources available online
• mostly dedicated to software development (dev)
  • quickly gets too complicated for basic research use (individual user or very small teams)
  • online tutorials rarely cover the parallel management of code AND data files
• “setup” can be a little technical, we will help get you started

Installing git

• linux (Debian/Ubuntu):

sudo apt install git

• Windows? -> https://gitforwindows.org

Git at INRAE

• Git = version control software
• “Github” is a developer plateform
• “Gitlab” is an open-source software to run your own developer plateform
• We use an instance of Gitlab installed on INRAE servers (open-source)

Setup 1: Login in!

• connexion SSO (https://forgemia.inra.fr)
• choose “INRAE”
• use your ldap to connect

Home page

Setup 2: Create your personnal access token

This will be the password you use to connect when you work from the terminal (i.e. when you perform actions sur as “push” and “pull”)

• click on you picture or avatar (initials? top-right)
• go to “Edit profile”
• then in the bar on the left “Access Tokens”

Setup 2: Create your personnal access token

• Add new token
• choose a name
• select all tick boxes (not all are necessary but doesn’t matter)
• remove expiration date if it’s there (and if it let’s you. )
• Click on “Create personal access token”

Create your first repository

• Click on “+” and “New project/repository”
• “Create a blank project”

Create your first repository

Create your first repository

• select your user name in “Pick a group or namespace”
• choose a name for your project -> initiation_git_bbrachi
• chose level of visibility (Private)

Create your first repository

• deselect “Initialize repository with a README” (set as default)

Your first repository is created!

Upload files from your computer

Setting up your repository

git config --global user.name "Benjamin Brachi"
git config --global user.email "benjamin.brachi@inrae.fr"

Push an existing folder

• Move to the folder we created earlier

cd ~/sandbox/myinternship/initiation_git

• initialize git in the folder

git init --initial-branch=main

(creates a hidden “.git” folder)

• Configure your local repository with the adress of the remote location

git remote add origin git@forgemia.inra.fr:bbrachi/initiation_git_bbrachi.git

Push an existing folder

• Add all the files (i.e. “stage” them)

git add .

• commit your changes to the repository

git commit -m "Initial commit"

Push an existing folder

git push --set-upstream origin main

Adding files and working with Gitlab

• An important file in a repository is the “README” file.

• We will create it now in our folder.

• In the terminal type:

touch README.md
gedit README.md

• Or create it with your favorite text editor (not Word!)

Writing the README file

In the README.md file let’s write something of the form:

# Git initiation course in BIOGECO

## Contributors
Where you write who's working on this project.

## Description
This is a test repository to practice using version control. 

## Content
Maybe a table of content of this file if it is long. 

## Requirements
What is needed to run this analyses in addition to what's in the repository? 

Adding the README.md file to the repository

git status

Adding the README.md file to the repository

• Staging:

git add README.md

• Committing changes with a short description:

git commit -m "adding the README file"

• Pushing to the remote repository!

git push

This password is a pain… Let’s fix that

• Setting up the SSH key means you don’t need to type your password for pushing/pulling.
• Keys function by pair: one local (private), one remote (public).

Before we create a key, let’s see if you have one

• Go to .ssh/in your home folder (hidden directory, ls -a to see it)

ls -a ~/

It’s not there? -> You don’t have keys.

• In Windows, probably you don’t, if you do, you’d know…

Let’s create a ssh key pair

ssh-keygen -t ed25519 -C "benjamin.brachi@inrae.fr"

• press enter
• check the file name. If you didn’t have keys the default is fine.
• if you did, maybe you want to add a little something to the name to recognize it.
  • I used /Users/bbrachi/.ssh/id_ed25519_gitinit
• skip the passphrase (simply press enter twice)

Let’s create a ssh key pair

• This procedure produces two files in the .ssh folder:
  • id_ed25519_gitinit
    • the local (private) end of the key pair (decodes)
  • id_ed25519_gitinit.pub
    • the public key (encodes), the one you put in gitlab (or on the cluster)

Add it to your gitlab account

Copy it to the clipboard:

• UNIX:

xclip -sel clip < ~/.ssh/id_ed25519_gitinit.pub

• Windows (in the Git Bash):

cat ~/.ssh/id_ed25519_gitinit.pub | clip

Let’s check if the ssh key works

ssh -T git@forgemia.inra.fr

If it is the first time you connect, you will see:

The authenticity of host 'gitlab.example.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)?

Type “yes” and enter.

It should say “Welcome to gitlab, username!”

Now let’s pratice add/commit/push again

## Inserting an image:

![An image from the internet](https://vickysteeves.gitlab.io/repro-papers/img/final-doc.jpg)

git add README.md

git commit -m "adding an image to the README file"

git push

Managing code and data files: the .gitignore file

# Create a gitignore file in order to untrack all files ending with the following extensions:
printf ".Rproj.user\n*.Rproj\n*.mp4\n*.fastq\n*.xlsx\n*.docx\n" > .gitignore

How to share large data files with collaborators?

What file size are we talking about?

• Less than few hundred kb ==> can be synchronized using git (but beware not to synchronise all your unnecessary files).
• More than 1Mb ==> you must use another storage to share this file (dataverse, EBI, NCBI) and document how to retrieve it in the README.md.

What did we learn?

Branches

Why create a branch?

• The analyses I have done up to now are solid/robust
• I have a couple of options for the next steps but I’m not sure which one I should do.
• I’ll try one way and see how it goes.

Why create a branch?

• My supervisor invites me to his repository (which has the basis for my work)
• During my intership I will do new analyses. I create my own branch.
• My supervisor may continue working on another branch as well.

Let’s play a game

• team-up with your neighbour
• one will be the supervisor
• the other will be the student

Supervisors: invite your student to your repository!

Students: retrieve your supervisors repository

cd ../ ## to leave your own working repository
git clone git@forgemia.inra.fr:bbrachi/initiation_git_lduva.git

cd initiation_git_lduva

Pull before you start working!

Files and branches in a collaborative projects are subject to changes by your collaborators, or by yourself on another computer for example.

In Practice…

1. Supervisors: make a change to the Rmd file, then add, commit and push to the main branch (like we did before)

2. Students: in your local copy of your supervisors repository pull the latest changes:

git status
git pull

Now we work with branches

git checkout -b student
git pull

Basculement sur la nouvelle branche 'student'

Connecting local branches to the remote repository

This branch is local!

Connect it to remote branch with :

git push --set-upstream origin student

git branch --all

Let’s work in our new branch

• Student and supervisors, go to Rstudio, and add a revolutionary piece of code to make a very nice figure in initiation_git.Rmd file

• Supervisors have too many things to think about and forget to capitalize the S of Species!

Let’s work in our new branch

• Because the students are smarter, they also indicate in the README.md that the R packages ggplot2 and gally are required for this project to work.

• (close the Rmd and README files when you’re done)

• Both students and surpervisors add,comit and push changes and check that all is up to date with git status

Push the changes to our new branch and see how the graph looks

• Stage or add the changes

git add initiation_git.Rmd
git add README.md ## for students only

• committing changes with a short description:

git commit -m "added an plot of Iris data using gally"

• Pushing to the remote repository!

git push

Merge into one version

• Before the end of the internship, the supervisor needs to merge the students work back into his own.

• Supervisors you will merge the student branch to the supervisor branch.

1. git status again to make sure all is up to date on your branch
2. Make a local copy of the student branch

git pull
git checkout student

Merge into one version

3. Switch back to your branch

git checkout supervisor
git merge student

And it spits out:

Fusion automatique de initiation_git.Rmd
CONFLIT (contenu) : Conflit de fusion dans initiation_git.Rmd
La fusion automatique a échoué ; réglez les conflits et validez le résultat.

This means there are conflicts.

• Look at the README.md. How does it look? Which version is it?
• Look at the Rmd file?

Conflicts in merge

The Rmd file should look something like this:

• conflicts are highlighted with >>>>>>> and =======

Conflicts in merge

• HEAD means you, what it was in your branch
• “student” is the branch you were trying to merge into your branch.
• Simply edit to file to keep the version you want (student version).
• then add, commit, and push as usual.

git add initiation_git.Rmd
git commit -m "resolved merge conflicts in Rmd"
git push

What we’ve learned

Additional commands

git stach
git stach pop
git cherry-pick
git commit --amend ## change a commit not pushed