1
00:00:00,560 --> 00:00:06,720
Hello and welcome to the fourth video of our 
GROOPS tutorial video series. This time we want  

2
00:00:06,720 --> 00:00:12,320
to focus on loops and conditions and I'm going to 
show you how you can use them to make your script  

3
00:00:12,320 --> 00:00:19,200
even more powerful and even more flexible. So 
we are using our script from the previous video,  

4
00:00:19,200 --> 00:00:26,240
where we visualized differences between kinematic 
and reduced-dynamic orbits for certain satellites.  

5
00:00:28,560 --> 00:00:32,960
If we go back to our data, you can see that 
we have data for three satellites: GRACE-1,  

6
00:00:32,960 --> 00:00:39,760
GRACE-2 and TerraSAR-X. And we have data 
for one week for each satellite. Of course  

7
00:00:40,400 --> 00:00:44,880
we could run our script for 
each satellite and each day,  

8
00:00:44,880 --> 00:00:49,280
but that's cumbersome, so it would be great if 
we could run our script once and it would plot  

9
00:00:51,040 --> 00:00:56,640
the differences for each day and each 
satellite. Thankfully, we can do this using  

10
00:00:58,000 --> 00:01:04,320
loop feature in GROOPS. First of all, we want to 
loop over our dates, so we'll add a new program  

11
00:01:05,200 --> 00:01:08,800
called "LoopPrograms", and here we 
can define a loop. There are different  

12
00:01:09,680 --> 00:01:15,760
options: You can use times, you can loop from 
a file, from a matrix and so on. In our case,  

13
00:01:15,760 --> 00:01:27,360
we will use "timeIntervals" and we will define our 
time period so it starts on 2015-01-01 and ends on

14
00:01:27,360 --> 00:01:33,600
2015-01-08. Here, since 
we're using "timeIntervals",  

15
00:01:34,400 --> 00:01:41,600
the end point of the interval has to be one past 
the last day because intervals are always defined  

16
00:01:41,600 --> 00:01:47,600
between two time points. And for sampling we use 
"1", so we want a daily sampling. For convenience,  

17
00:01:47,600 --> 00:01:53,840
we will set both "timeStart" and "timeEnd" 
as global variables by right-clicking them,  

18
00:01:54,480 --> 00:02:02,000
so we can change them in the global section. We 
don't have to go to this location to change them.  

19
00:02:04,080 --> 00:02:09,120
What we do not need anymore is the "time" 
variable we used in the previous video  

20
00:02:09,120 --> 00:02:18,160
because this loop creates a new variable called 
"loopTime" – we can use a different name but  

21
00:02:19,120 --> 00:02:27,520
let's stick with "loopTime" – and in each 
iteration this variable contains the current  

22
00:02:28,160 --> 00:02:34,034
timestamp. For example, in the first iteration it 
will be the 2015-01-01, then 2015-01-02 and so on.  

23
00:02:36,400 --> 00:02:40,720
What we can also set here is if we want to 
continue after an error. If any of the programs  

24
00:02:40,720 --> 00:02:46,640
within this loop cause an exception, we can say, 
okay, just skip to the next iteration and continue  

25
00:02:46,640 --> 00:02:53,840
going on. So we'll set that to "yes" just to 
be sure. So this is our outer loop over the  

26
00:02:54,400 --> 00:03:01,200
days. We also want to add an inner 
loop over the satellites. In this case,  

27
00:03:01,200 --> 00:03:07,120
we're going to use a "manualList" so we can 
define a list of strings. This will be "GRACE-1",

28
00:03:09,600 --> 00:03:10,960
"GRACE-2" and

29
00:03:14,320 --> 00:03:22,960
"TerraSAR-X". And in this case, we will call 
our variable "satellite". We already have a  

30
00:03:22,960 --> 00:03:27,680
global variable "satellite", which we used in the 
previous video. We don't need that anymore because  

31
00:03:27,680 --> 00:03:32,480
now our loop will define the variable "satellite" 
and in the first iteration it would be "GRACE-1",  

32
00:03:32,480 --> 00:03:38,480
then "GRACE-2" and then "TerraSAR-X". Again, 
we'll set "continueAfterError" to "yes" because  

33
00:03:39,040 --> 00:03:45,280
if there is no data on one day, for example, we 
still want to visualize the other days. Okay,  

34
00:03:45,280 --> 00:03:50,480
so now we have our two loops: We have the 
outer loop over the days and the inner loop  

35
00:03:50,480 --> 00:03:55,920
over our satellites. For convenience's sake, we 
can set the whole loop – the inner loop – global  

36
00:03:55,920 --> 00:04:01,760
and let's call it "loopSatellite". Now 
in our global section I can easily change  

37
00:04:01,760 --> 00:04:07,120
the list of satellites. I can add some or 
disable some, so it's just for convenience.  

38
00:04:08,640 --> 00:04:16,640
Okay and now we have to move our programs into the 
loop. We can just copy or move them one by one.  

39
00:04:17,280 --> 00:04:25,600
Actually, for "FileCreateDirectories": We would 
keep creating the plot directory outside of the  

40
00:04:25,600 --> 00:04:31,280
loop because we don't really want to create this 
directory once, but the "tmp" directory we will  

41
00:04:31,280 --> 00:04:41,840
create in each loop. More on that in a 
second. Let's continue moving our programs.

42
00:04:44,640 --> 00:04:49,120
Okay, so now I've moved all the programs we 
had on the outside before into the inner loop.  

43
00:04:49,760 --> 00:04:55,840
So all of these programs will be 
run for each satellite on each day.  

44
00:04:58,000 --> 00:05:06,080
As I said – or what we should now consider is 
that we we might use different "tmp" directories  

45
00:05:06,080 --> 00:05:15,360
for each loop. Just to be safe, so that we 
do not use any previous files from previous  

46
00:05:15,360 --> 00:05:23,840
iterations accidentally. So what we can 
do is we can say we want to replace –

47
00:05:24,400 --> 00:05:29,040
it's Ctrl+F to go to Find & Replace – we 
want to replace all occurrences of "tmp" with  

48
00:05:29,040 --> 00:05:34,560
a global variable "{tmpDir}". Let's 
replace all. With that button you can  

49
00:05:36,000 --> 00:05:41,680
close all programs again. If we 
now add a global variable "tmpDir"  

50
00:05:43,680 --> 00:05:50,640
of type "filename" we can define a temporary 
directory we want to use for each iteration. Let's  

51
00:05:50,640 --> 00:06:05,600
use "tmp/{satellite}_{loopTime:%D}", so for each – 
if we now go to "FileCreateDirectories" – it will  

52
00:06:05,600 --> 00:06:12,880
create a temporary directory in the subdirectory 
"tmp" with the name of the satellite, underscore,  

53
00:06:12,880 --> 00:06:21,120
and the date parsed as an ISO date string. Which 
is convenient because then each day and satellite  

54
00:06:21,120 --> 00:06:25,280
will have their own temporary directory and 
there can be no confusion with the data.

55
00:06:27,520 --> 00:06:34,800
Okay, so far so good. One thing we still 
have to change is – if you remember, we  

56
00:06:34,800 --> 00:06:44,080
used our variable "time" beforehand and, here for 
example, and now, of course, we don't want to use  

57
00:06:45,440 --> 00:06:52,800
"time" but instead our loop variable 
"loopTime". So we're going to replace that  

58
00:06:53,440 --> 00:06:56,560
and now we should be able to run this script.

59
00:06:59,760 --> 00:07:03,840
As you can see it runs. If you go to our 
directory and go into the "plots" directory,  

60
00:07:03,840 --> 00:07:07,520
you can see, okay, there's a GRACE-1 
plot, GRACE-2 plot, now a TerraSAR-X  

61
00:07:07,520 --> 00:07:13,840
plot and then for the second date a GRACE-1 plot 
and so on. The program will continue running.  

62
00:07:14,960 --> 00:07:21,680
It will take some time, of course, 
because we are creating quite a few plots.  

63
00:07:24,400 --> 00:07:33,840
Let's delete them for now because now I can show 
you another feature, which is we can parallelize  

64
00:07:33,840 --> 00:07:40,720
our script, so we can run our script in parallel. 
So far we ran it with a single process but if we  

65
00:07:40,720 --> 00:07:46,640
have tasks that can be run on multiple cores 
in parallel then this can be quite useful.  

66
00:07:47,920 --> 00:07:53,520
In this case, it's pretty easy. Let's just 
say we want to parallelize our time loop, so  

67
00:07:53,520 --> 00:08:01,200
we want to plot multiple dates at the same time. 
This is very simple because we can, here, adjust  

68
00:08:01,200 --> 00:08:07,840
in the outer loop the "processCountPerIteration" 
from "0" to "1". "0" means all processes are used  

69
00:08:08,560 --> 00:08:14,960
for each iteration, so all the programs use all 
processes. In this case, this doesn't make much  

70
00:08:14,960 --> 00:08:20,560
sense because none of these programs can benefit 
from more than one process. But if we change this  

71
00:08:20,560 --> 00:08:29,280
to "1" in this loop, only one process will be used 
for each iteration in this loop, so for each day.  

72
00:08:30,560 --> 00:08:38,960
If we now run our script, not with the single 
process executable but instead with the  

73
00:08:38,960 --> 00:08:49,600
MPI (Message Passing Interface) executable. If you 
have installed MPI, you can compile a multi-core  

74
00:08:50,480 --> 00:08:56,000
or multi-process executable and we can run 
this instead. And if we run this, we'll see,  

75
00:08:56,000 --> 00:09:00,960
okay, multiple processes are working at 
the same time and you can see multiple  

76
00:09:00,960 --> 00:09:07,840
plots – or all three plots – are generated at 
the same time. So it goes quite a bit faster.

77
00:09:09,280 --> 00:09:17,440
What we can also do is – if you think this 
is useful for your script – we can write  

78
00:09:17,440 --> 00:09:23,920
log files. We could write a log file 
if we check this here. Then it writes  

79
00:09:23,920 --> 00:09:30,720
all the output into this log file. But another 
thing we could do is add a program called  

80
00:09:31,360 --> 00:09:41,100
"GroupPrograms" and if we move our inner loop 
into this program and define an "outputfileLog"  

81
00:09:41,120 --> 00:09:47,120
for this group – which we could say it should 
be in a directory "log" and should be...

82
00:09:52,040 --> 00:10:00,240
... should use the date as a file name. We have to 
be careful: We have to create our log directory at  

83
00:10:00,240 --> 00:10:08,640
the beginning. But now what happens is that for 
each outer loop – for each day – it creates a  

84
00:10:08,640 --> 00:10:15,200
log file where the output from this inner loop 
is written to. If I run this again with MPI,  

85
00:10:16,560 --> 00:10:21,520
again, everything happens in parallel but 
now we have our log directory and we have  

86
00:10:21,520 --> 00:10:26,640
one file per day. And this file contains 
the log output of the inner loop, so  

87
00:10:27,200 --> 00:10:31,840
the conversion or the processing 
of the three satellites.

88
00:10:33,600 --> 00:10:37,440
And one more thing I want to show 
you is we can also add a condition.  

89
00:10:38,320 --> 00:10:42,640
For example, if you remember, we 
said in our script that we want to  

90
00:10:43,840 --> 00:10:49,520
visualize outliers by black diamonds in 
the plot. And we could now say we only  

91
00:10:49,520 --> 00:10:55,040
want this to happen for GRACE satellites. So if 
I right-click this layer for the diamonds and  

92
00:10:55,040 --> 00:11:03,467
say "Set Condition", I can define a new condition 
– let's call it "conditionIsGrace" – and  

93
00:11:04,080 --> 00:11:08,080
for this condition there are multiple types 
you can use. You can check files, you can  

94
00:11:08,080 --> 00:11:13,840
run a command, and so on. But also you can 
match strings and patterns so I'll choose here  

95
00:11:14,640 --> 00:11:19,360
"stringContainsPattern". The string 
we check will be the content of  

96
00:11:19,360 --> 00:11:24,640
our "satellite" variable and the pattern 
will be "GRACE". So if "GRACE" is contained  

97
00:11:26,000 --> 00:11:33,600
in the string, the condition will be true. We 
have set this on this layer, so if we now run this  

98
00:11:35,600 --> 00:11:43,120
and check our plots, we should see in 
the GRACE plots there still are the  

99
00:11:44,320 --> 00:11:48,480
black diamonds – there are outliers 
– but if I go to a TerraSAR-X plot,  

100
00:11:48,480 --> 00:11:55,840
you can see there are no more outliers 
here. This is one way to use a condition.  

101
00:11:57,200 --> 00:12:04,320
I want to show you another way, or an alternative, 
for both conditions and loops because as you can  

102
00:12:04,320 --> 00:12:09,440
see, for loops I used the program "LoopPrograms" 
and for conditions I right-clicked and added  

103
00:12:09,440 --> 00:12:15,840
it. You can also do the reverse. Let's 
disable this program for now and add...

104
00:12:21,360 --> 00:12:27,520
... add a new program called – again, let's 
use "LoopPrograms". This time we will loop  

105
00:12:27,520 --> 00:12:34,480
over our satellites and here we now are 
going to use the program "IfPrograms".  

106
00:12:34,480 --> 00:12:41,280
So it's just an alternative way of using 
conditions. We will use our "conditionIsGrace"  

107
00:12:41,280 --> 00:12:49,400
in this case and in this program – 
just to show you what I want to do –

108
00:12:50,720 --> 00:12:53,440
I'm going to use the program 
"InstrumentConcatenate", which,  

109
00:12:53,440 --> 00:13:00,320
as the name says, concatenates instrument 
files. I can just quickly show you  

110
00:13:00,320 --> 00:13:08,000
that I can, for example, concatenate the 
orbit files – the original orbit files.

111
00:13:14,080 --> 00:13:20,320
Let's use some output file. What I can do now 
is – here you can see I have one input file but  

112
00:13:21,840 --> 00:13:27,520
this element contains the ellipses, which means 
you can provide multiple files. You could now copy  

113
00:13:27,520 --> 00:13:34,080
and paste it and change the date but you can also 
set a loop on this element. Beforehand, we have to  

114
00:13:34,640 --> 00:13:40,320
go back to our previous loop, where we defined 
the loop over the intervals. We can set that  

115
00:13:40,320 --> 00:13:48,880
global – let's call it "loopIntervals". 
Now we can use this loop here. We can say  

116
00:13:49,920 --> 00:13:57,040
"Set Loop", "loopIntervals", and now this 
config element will be duplicated for each  

117
00:13:57,040 --> 00:14:03,520
entry in this loop, so in this case 
for the days. If I now run this program  

118
00:14:05,280 --> 00:14:11,600
– let's use the default one – you can 
see that it is run twice because we have  

119
00:14:11,600 --> 00:14:15,760
the condition that it's only run for GRACE 
satellites – GRACE-1 and GRACE-2 – and then  

120
00:14:15,760 --> 00:14:20,880
the program "InstrumentConcatenate" reads 
all the seven files and concatenates them.  

121
00:14:22,400 --> 00:14:26,160
Okay, so these are the basics 
of our loops and conditions.  

122
00:14:26,160 --> 00:14:31,840
Of course, this is a very powerful feature, 
so you can do quite a lot of things and add  

123
00:14:34,080 --> 00:14:42,750
complex behavior to your script. Yeah, so that's 
it for now, thanks for watching and bye bye!