13:24:10 From Bianca De Saedeleer : yes 13:24:14 From Matthieu Salamone : yes 13:24:18 From Giovanna Roda : Did everyone manage to download the slides# 13:24:23 From Milena Veneva : yes 13:24:24 From João Especial : yes 13:24:28 From Pedro Xavier Ramos : yeah 13:24:36 From Teslim O : yh 13:24:38 From Maha Dessokey : yes 13:31:47 From Betzabeth Leon : yes 13:31:56 From Edixon Parraga : yes 13:32:22 From Patricio Puchaicela : yes 13:32:35 From Zacarias Benta : rotfl 13:38:21 From Matilde Pato : yes 13:38:24 From Teslim O : yh 13:38:33 From Betzabeth Leon : yyes 13:38:47 From Zacarias Benta : We can see everything 13:38:50 From Giovanna Roda : Can you see the picture of HDFS? 13:38:55 From Patricio Puchaicela : yes 13:38:56 From Milena Veneva : y 13:38:59 From Betzabeth Leon : y 13:39:31 From Edixon Parraga : Y 13:42:36 From Maha Dessokey : y 13:42:42 From Bianca De Saedeleer : yes 13:42:46 From Bram De Jaegher : y 13:43:18 From FABIANA MIRANDA : yes 13:43:26 From Emre Perihan : I have 13:43:27 From Zacarias Benta : I’ve got a terminal and have downloaded your files into a Hadoop folder. 13:43:32 From Nuno Agostinho : yes 13:43:36 From Betzabeth Leon : yes 13:43:40 From Maha Dessokey : yes we have a terminal and downloaded the files 13:43:42 From Leon Kos : Please open the terminal on your NoMachine 13:43:44 From Leon Bogdanovic : Yes. 13:43:49 From Pedro Xavier Ramos : Yes, I have the terminal 13:44:10 From Leon Kos : System->konsole 13:45:17 From Matthieu Salamone To Leon Kos(privately) : I have an issue connecting to the HPC 13:45:33 From Leon Kos : module avail hadoop 13:45:41 From Giovanna Roda : You can find all commands in the file /home/campus00/public/useful_commands 13:45:47 From Matthieu Salamone To Leon Kos(privately) : following the instructions on making a connection with "login.hpc.fs.uni-lj.si" 13:46:05 From Matthieu Salamone To Leon Kos(privately) : but when trying to connect I get a "invalid credential error" 13:46:15 From Matthieu Salamone To Leon Kos(privately) : using the credential that were provided to me 13:46:37 From Leon Kos To Matthieu Salamone(privately) : viz.hpc.fs.uni-lj.si is the same. 13:46:54 From Giovanna Roda : Is someone able to see the file with the commands? 13:46:58 From Matilde Pato : This information is in website: https://events.prace-ri.eu/event/1029/timetable/ 13:47:11 From Giovanna Roda : useful_commands.txt 13:47:20 From Matilde Pato : Yap, and all examples 13:47:22 From Leon Kos To Matthieu Salamone(privately) : Which campus? Please write to alenka.maffi@lecad.fs.uni-lj.si to get another account 13:47:30 From Matthieu Salamone To Leon Kos(privately) : oke 13:48:09 From Leon Kos : module load Hadoop/2.6.0-cdh5.8.0-native 13:48:55 From Bram De Jaegher : Has anyone figures out how to switch the keyboard layout to azerty? 13:49:14 From Bianca De Saedeleer : I have the same problem 13:49:21 From Bram De Jaegher : I'm on the HPC 13:49:25 From Leon Kos : hdfs getconf -namenodes 13:49:57 From Bram De Jaegher : What should I show? 13:50:09 From Marta : The output of the command. I can do it 13:50:36 From Marko Prelevikj : yes, we can hear you 13:50:36 From Numan Nusret Usta : hear you 13:50:38 From LECAD PRACE PTC : If you have any questions, you can write, it is easier. 13:51:34 From Bianca De Saedeleer : But is there a way to chance the keybord? Hpc is in qwerty but my own keyboard is in azerty 13:51:41 From Bianca De Saedeleer : * change 13:53:58 From Leon Kos : To change the keyboard use Settings->KPersonalizer 13:55:16 From Bram De Jaegher : Yes but there are only US English and slovenian as language settings. neither of them use azerty 13:55:38 From Leon Kos : Or use icon at the bottom right. 13:59:37 From Bianca De Saedeleer : I've found the azerty setting, thank you! :) 14:00:21 From Bram De Jaegher : Me too 14:00:45 From Leon Kos : cat /home/campus00/hadoop/useful_commands 14:01:01 From Giovanna Roda : Who wants to share the screen for this exercise? 14:01:08 From Miguel : I can share 14:06:45 From Matthieu Salamone To Leon Kos(privately) : problem was resolved 14:08:01 From Leon Kos To Matthieu Salamone(privately) : OKay. How. 14:08:25 From Matthieu Salamone To Leon Kos(privately) : new credentials from Alenka 14:09:09 From Leon Kos To Matthieu Salamone(privately) : SO which campusXX was that problematic? 14:09:58 From Matthieu Salamone To Leon Kos(privately) : my initial credentials had no campusXX in the username: it was user : 29 password : NjU0YzVk 14:10:03 From Matthieu Salamone To Leon Kos(privately) : maybe that was the issue 14:10:23 From Leon Kos To Matthieu Salamone(privately) : shoud be campus29, of course. 14:10:38 From Matthieu Salamone To Leon Kos(privately) : That would explain why it didn't work 14:10:44 From Matthieu Salamone To Leon Kos(privately) : thanks for support 14:11:33 From Leon Kos To Matthieu Salamone(privately) : So compus29 is OK still. I would say. 14:11:45 From Matthieu Salamone To Leon Kos(privately) : i'll try during coffee break 14:12:32 From Leon Kos To Matthieu Salamone(privately) : It is campus29 and not compus29 mistyped. 14:13:36 From Leon Kos : I can answer some HPC and usage problems now. If they are any, of course. 14:25:22 From Leon Kos : more or less /home/campus00/hadoop/useful_commands 14:32:43 From Giovanna Roda : Use the option -D mapreduce.task.io.sort.mb=100 14:33:19 From Giovanna Roda : hadoop jar $STREAMING_JAR -D mapreduce.task.io.sort.mb ... 14:38:31 From LECAD PRACE PTC : Break until 3 p.m. 14:38:42 From Leon Kos : Break. Starting at 3pm 14:59:33 From Miguel : In order to solve the memory error, just run the command above 15:00:34 From Emre Perihan : in this case I'm getting another error 15:00:36 From Emre Perihan : ERROR streaming.StreamJob: Error Launching job : Output directory hdfs://viz.hpc:54310/user/campus27/simplest-out already exists Streaming Command Failed! 15:00:44 From Giovanna Roda : From now on we always need to use Hadoop jar $STREAMING_JAR -D mapreduce.task.io.sort.mb=100 … 15:00:52 From Marko Prelevikj : you have to delete the simplest-out directory as well 15:00:56 From Giovanna Roda : otherwise we get an OOM error 15:00:59 From Milena Veneva : Delete the simplest-out 15:01:05 From Marko Prelevikj : hadoop fs -rm -r simplest-out 15:01:07 From Miguel : yes, you need to delete the 'simplest-out' dir created in the failed attemp 15:01:13 From Giovanna Roda : Yes 15:01:30 From Bianca De Saedeleer : I get the error "not a valid JAR: /home/campus12/-D" 15:01:49 From Giovanna Roda : We are starting again 15:02:28 From Leon Kos : -D shoud have space 15:02:32 From Emre Perihan : thank you Miguel and Marko! I've solved ıt 15:02:48 From Marko Prelevikj : run this as well: export STREAMING_JAR=/opt/apps/software/Hadoop/2.6.0-cdh5.8.0-native/share/hadoop/tools/lib/hadoop-streaming-2.6.0-cdh5.8.0.jar 15:10:23 From Giovanna Roda : Were you able to copy the mapper.py file to your directory 15:10:27 From Giovanna Roda : ? 15:10:35 From Andrew Walakira : no 15:10:37 From Miguel : yes 15:10:38 From Leon Kos : cp /home/campus00/public/*.py . 15:10:41 From Patricio Puchaicela : yes 15:10:49 From Maha Dessokey : i used wget https://events.prace-ri.eu/event/1029/sessions/3276/attachments/1310/2290/reducer.py 15:11:12 From Bianca De Saedeleer : No.. 15:11:22 From Zacarias Benta : Just access the course page on the cluster and download the files 15:11:41 From Giovanna Roda : Bianca cp /home/campus00/public/*.py . 15:12:15 From Bianca De Saedeleer : I'm getting an eroor in slovenian 15:12:21 From Bianca De Saedeleer : * error 15:12:32 From Bianca De Saedeleer : cp: manjkajoč operand ciljne datoteke za »home/campus12/public/mapper.py« Poskusite »cp --help« za izčrpnejša navodila 15:12:48 From Leon Kos : Missing space dot at the end 15:13:31 From Bianca De Saedeleer : I cannot see your screen now 15:14:17 From Leon Kos : You should. 15:15:19 From Leon Kos : Which campus?? Bianca. 15:15:22 From Bianca De Saedeleer : 12 15:16:00 From Leon Kos : Please allow connection from leon to your screen shortly. 15:16:18 From Bianca De Saedeleer To Leon Kos(privately) : Hi, while i was sharing my screen I could not see the zoom 15:16:37 From Leon Kos To Bianca De Saedeleer(privately) : oh yes. 15:17:17 From Leon Kos To Bianca De Saedeleer(privately) : Slash befor home 15:17:37 From Leon Bogdanovic : This shoould work: cp -r /home/campus00/public/*.py ./ 15:18:12 From Leon Kos To Bianca De Saedeleer(privately) : You are OK now. 15:18:17 From Bianca De Saedeleer To Leon Kos(privately) : Okay, thank you! 15:18:50 From Leon Kos To Bianca De Saedeleer(privately) : I will disconnect now. But you can ask me for a look anytime I am free. 15:19:01 From Bianca De Saedeleer To Leon Kos(privately) : Thank you very much! 15:19:44 From Leon Kos To Bianca De Saedeleer(privately) : Weird keyboard for me. Good luck. 15:20:36 From Bianca De Saedeleer To Leon Kos(privately) : Haha, it is azerty. We use that in Belgium 15:23:10 From Numan Nusret Usta : how can we get the wiki321 file 15:25:30 From Numan Nusret Usta : to the hpcfs 15:25:58 From Leon Kos : ls /home/campus00/public 15:30:17 From Leon Kos : hadoop fs -put /home/campus00/hadoop/data/wiki321MB 15:36:35 From Giovanna Roda : What is the most frequent word? 15:36:46 From Bianca De Saedeleer : the 15:36:53 From Milena Veneva : the 15:37:04 From Bianca De Saedeleer : Haha 15:37:15 From Bianca De Saedeleer : No, I managed to do it 15:39:32 From Miguel : yes 15:39:39 From Leon Kos : cp /home/campus00/hadoop/swap_keyval.sh . 15:43:28 From Milena Veneva : Maybe share the prompt? 15:55:23 From Leon Kos : export STREAMING_JAR=/opt/apps/software/Hadoop/2.6.0-cdh5.8.0-native/share/hadoop/tools/lib/hadoop-streaming-2.6.0-cdh5.8.0.jar 15:56:54 From Giovanna Roda : hadoop jar ${STREAMING_JAR} -D mapreduce.job.output.key.comparator.class=org.apache.hadoop.mapred.lib.KeyFieldBasedComparator -D mapreduce.partition.keycomparator.options=-nr -D mapreduce.task.io.sort.mb=100 -file swap_keyval.sh -input wordcount_out_3 -output wordcount_out_4 16:08:09 From Giovanna Roda : Any questions 16:18:26 From FABIANA MIRANDA : are you going to upload the records? 16:18:58 From Giovanna Roda : Yes 16:19:06 From Pedro Xavier Ramos : Thanks! 16:19:11 From Patricio Puchaicela : Thanks 16:19:12 From Betzabeth Leon : perfect, thanks! 16:19:12 From FABIANA MIRANDA : Thanks! 16:19:12 From Maha Dessokey : Thanks alot 16:19:13 From Bianca De Saedeleer : Thank you! 16:19:30 From Mohammad Rahimi Gorji : Thanks a lot. 16:19:32 From Cymon J. Cox : Thanks 16:20:06 From Miguel : Thanks 16:20:07 From Betzabeth Leon : bye 16:20:10 From Emre Perihan : thank you