On Sunday 27 Jul, 2014
I’m feeling terrible - I forgot to blog post last week. Again. I’ll try to make up for it…
Software Carpentry sprint summary
The sprint lasted two days (22-23 July 2014) and it was a blast! Poland had wonderful representation in Kraków, where I’m living.
Most of the group worked on Data Carpentry lessons:
- writing new in Excel or Python+Pandas
- checking existing in Bash
As far as I know, everything was merged in! That’s what I call a success.
I was the only person working on a different project, ie. on my Pitt.
Most of the time I was working alone, but once or twice we’ve got together with Mozilla Paris to test some things.
I want to kindly thank Rémi Emonet and Raniere Silva for their help! These guys not only contributed feedback and bug reports, but also took part in coordinated calls with me. Thank you!
Having tested Pitt with Paris, I’m now more certain that the signalling part of my application is now very close to being bug-free. Or maybe it is already. I’m really liking how easy it is to add new features and still be certain, that the application won’t break accidentally.
Pitt progress during sprint
Here’s the list of everything that was achieved during the sprint:
- five hundred lines of documentation have been written
- fixed too high volume for the warning beep
- forced low video resolution
- dropped _pubsub part from directories or file names
- turned on TURN server for testing – and successfully tested with Mozilla Paris
Exciting features coming
For the next big update, I’m thinking about setting some huge goals.
- Mobile-friendly interface (at least for tablets) and fixes for non-full-HD devices (https://github.com/pbanaszkiewicz/pitt/issues/18)
- Testing (and fixing in case of errors) voice-only connections – that’s something I didn’t get to test during the sprint (https://github.com/pbanaszkiewicz/pitt/issues/19)
- Etherpad integration – as far as I am concerned, Etherpad doesn’t provide any way to share chat with other sites. That’s a bummer. While I don’t need Etherpad’s content integration, shared chat would be extra nice. This needs futher invesitagion (https://github.com/pbanaszkiewicz/pitt/issues/23)
- Chat: if not possible to integrate with Etherpad, write our own (https://github.com/pbanaszkiewicz/pitt/issues/20)
- Peer2Peer file sharing – something exactly like ShareFest (https://github.com/pbanaszkiewicz/pitt/issues/21)
- Hard: better broadcasting mesh (students act like relays / proxies), read more at https://github.com/pbanaszkiewicz/pitt/issues/22
I think the end of Google Summer of Code is nearing, but I like this project a lot, and I definitely will continue working on it.
On Monday 14 Jul, 2014
I missed one post last week and this one is already delayed by two days (compared to my typical writings that appeared on Saturdays).
Fortunately, I have good excuses.
First, I wanted to get something done. Second, I was buried under real-life super exhausting stuff like post-exams leisure. ;)
Okay, good news: Pitt has new interface:
I separated application logic code from application appearance code. Overall, the code should be easier to understand for newcomers and a lot easier to maintain.
The list of issues is updated and available here.
I’m also starting to think about Software Carpentry Sprint that takes place on July 22-23. I’ll post some update soon…
On Sunday 29 Jun, 2014
This is gonna be a short one. Because of exams, I have to study a lot. And because there was not much progress on Pitt during this week.
Current version is v0.3.1, and it’s tagged in git repository. I’m incorporating git-flow model.
What I wanted to get in was two major features:
- countdown before switch from group split to broadcasting mode,
- small groups of students of variable-defined size.
I can tell that I was successful and these features are now part of v0.3.1 release.
While testing with Greg and Mike at their office and me here in Kraków, I couldn’t connect to them and vice versa. Connections between Mike and Greg worked as intended.
This was strange, because I didn’t change the code so much that it’d break. More so: last week while testing with Greg alone it worked!
So I conducted few more tests:
- Pitt: Kraków ↔ Kraków
- Pitt: Kraków ↔ Toronto Mozilla Office
- Pitt older version: Kraków ↔ Toronto Mozilla Office
- Other WebRTC application: Kraków ↔ Toronto Mozilla Office
In the previous week, I’m guessing Greg worked from his home. We wanted to try Pitt in this configuration (Kraków ↔ Greg’s home), but due to holidays in Canada we were not able to.
From what I can tell, the issue lies in the client IP addresses resolution. Here’s a lengthy article about how hard it is to do WebRTC properly: http://www.html5rocks.com/en/tutorials/webrtc/basics/#toc-signaling
Anyway, I hopefully will be able to easily overcome this issue.
On Sunday 22 Jun, 2014
This week me and Greg Wilson, my mentor, we decided to test run Pitt. It was a success and, while revealing some annoying bugs, it made Greg really excited.
Current version is v0.3, altough I might have tagged it wrong in git. Fortunately it seems that tag was not published to the GitHub repository.
This is a quick recap on what Pitt is capable of right now.
- Users coming in and quitting: thanks to signalling, as soon as someone appears online, they’re being logged in and everyone can see them.
- Quick switch mode: the instructor can easily and very quickly switch modes from broadcasting (to all students) to work in small groups (students are split into pairs or groups of 3).
The code is decent. I presume there are some bugs with saved state of the application. For example, what to do if the instructor is broadcasting and someone joins in? What if another instructor joins in? What if there are two instructors and one starts broadcasting, should other instructors see their stream too?
During my test run with Greg we quickly found out one super annoying bug.
There’s a very interesting human behavior: people slow down their talk when they hear themselves with small delay. It’s really hard to overcome this basic reflex.
Back to the bug: we found out that local stream video (ie. the one where you can see yourself, because browser plays the video from your webcam) is not muted by default. Add the playback delay to the mix and as a result Greg was struggling to speak.
This bug was really easy to fix (in HTML5: <video src=”…” muted=”muted”>) and the fix is included in the recent code.
There was also issue with the state of the application - it somehow broke for Greg (when he was a student). I have yet to discover the actual bug.
Greg wants me to add these two new features to the upcoming release:
- Countdown. The instructor presses “back to broadcast mode” button and the switch doesn’t happen immediately. First there’s 30 seconds countdown so that no students are interrupted in the middle of the sentence.
- Variable group size. The instructor can specify if the students are split into pairs or groups of 3, or groups of 6. This task is more challenging than it seems: I have to ensure that no students are left alone (what is often the case when there’s odd number of students).
I’m planning to release next version, v0.3.1, on Friday 27.06.2014 or earlier.
I have 3 exams before Friday and 2 after it. Ouch…
On Saturday 14 Jun, 2014
I had one exam this week (Aparatura Automatyzacji, eng. Automation Systems) and five more in next 2-3 weeks, so it is quite challenging for me to find time for peer instruction right now.
I did, however, spend most of this weekend trying to make Pitt broadcasting mode for instructors.
It works! Proof below:
As you can see, the layout and design in general is very rough. And so is the code.
This code is mostly based upon multiple events being propagated through sockets.
But I really don’t like the design of these events, and, frankly speaking, I much more prefer a “pub/sub” (publication - subscription) architecture.
At this moment, when the instructor triggers an event, the server has to repropagate it to every student.
In “pub/sub” arch, however, there’s no repropagation in between.
- WAMP is not a NodeJS module, it’s a protocol
- it can be (and indeed is!) implemented in various languages
- therefore it’s a solid foundation for a quickly changing service.
What I dislike about WAMP is that at the moment there’s only one leading implementation: Autobahn. And I’m not yet sure if I want to drop NodeJS backend in the future, but if so, then there’s already fast Autobahn|Python.
On Saturday 07 Jun, 2014
In this week’s videoconference between my mentors, Greg Wilson and Mike Hoye, Software Carpentry‘s instructor David Rio, and me, we decided to temporarily drop my current codebase and what I’ve done with Licode and Erizo.
This has some interesting consequences.
Firstly, we can’t have broadcasting using MCU in teacher’s mode. Instead, in this mode, we’ll have to open P2P connections, what’s probably going to be quite challenging for teacher’s internet connection.
In teacher’s mode, the teacher broadcasts his stream to all of their students.
In student’s mode, students are split into smaller groups (6 people per group in real life, at most 4 people via Internet - more is less efficient) and talk with each other.
But for now we want to ship first version as soon as possible, so we’ll care about that later.
Secondly, we’ll have a lot clearer and “closer to the metal” code. Opening sockets and handling all data transfers will be much easier, and I like that fact a lot!
On my personal side of news, I’m currently preparing for exams (first one: next Thursday), which leaves me very little time for GSOC #sadface…
On Friday 30 May, 2014
Let me quickly reintroduce basic ideas and plans for Peer Instruction project.
Peer instruction concepts
The most important feature of Peer Instruction is the ability to, from the technical point of view, quickly switch between teacher and student modes.
Teacher mode is basicly a broadcast sent to every student in a classroom. This might be quite challenging: you can have many direct connections and suffer from capped bandwidth and high CPU usage.
The solution is to introduce additional server that helps to “spread” the stream to multiple students. It essentially looks like this:
What’s really good about this design? There’s an excellent open-source MCU available: Licode. I must admit Licode is a little too complicated to set up, but authors’ support and continuous development make it a great option if someone wants, for example, to host their own alternative to Google Hangouts or Skype.
As you can see above, the interface is pretty simple, but it works (at least for 1 student). I have yet to test it for more students, but I haven’t received a server account at Mozilla to do so.
This is the second mode of broadcasting. A very crucial from the point of science of peer instruction.
In this mode students are split into smaller groups so that they can talk with each other.
Right now I’m thinking how to implement this mode. One of the ideas is to create one Erizo Room for every group of students.
The way events are propagated within Erizo makes it really easy to subscribe to other students’ streams within one particular room. Using multiple rooms also helps manage all these smaller groups.
I should also mention that Erizo can create “full MCU” rooms - ie. rooms where streams are going through the server - and Peer-to-Peer rooms, where the server only handles sessions, signalling and events, but the streams are actually transferred between the room participants.
I think for small groups of 2-4 students it’s a good idea to use P2P rooms and save resources on the server.
Unfortunately, this solution has some drawbacks, too. I found out that Erizo can only handle so much rooms at once (it’s specified in configuration, but I don’t know the retionale behind it).
My main focus for next week(s) is working PoC of students mode. Right now I’m working on a protocol used by Licode events.
It’s possible to send data through Erizo streams. I want to use this “channel” to communicate with students’ browsers.
For now I’ll only send data like “join room ABC for small group discussion” and “leave room ABC”. In future Peer Instruction might be more advanced - quite important feature is chat, and I will likely leverage the existing protocol for that.
Anyway, that’s the update status on my project. I highly welcome any contribution or comments.
Here’s project GIT repository: https://github.com/pbanaszkiewicz/peer-instruction
On Thursday 24 Apr, 2014
Peer instruction is a teaching methodology developed over 20 years ago. It’s different from typical learning classes in today’s colleges or universities. The basic workflow goes more or less like this:
- Teacher asks a question.
- Students individually answer it.
- Students are split into small groups to discuss their answers.
- Teacher reveals the good answer.
The key feature of this methodology is splitting students into small groups.
|||Peer tutoring is ranked as having 0.55 influence on student’s achievement, while 0.0 is none.|
If you take an online classes by Udacity, Coursera, Khan Academy or whoever there is, you may notice they don’t deploy peer instruction.
But with web technologies of today, it is possible to create a virtual classroom that leverages this promising teaching methodology! And my task is to do it.
The minimum outcome
No, I don’t want to create a “full-stack” virtual classroom. The minimum viable product of my work will be a proof-of-concept working web service that lets you quickly switch from broadcasting (or “teacher mode”) to many few-to-few multiplexed broadcasts (or “small-groups-talk mode”).
Of course, if I can, I’ll implement as many additional features as possible.
The only two difficulties I foresee right now are:
- The lack of fast MCU technology.
- The lack of fast internet.
MCU stands for “Multipoint Control Unit”, a software or hardware used to bridge media transmissions.
The only one open source software MCU for WebRTC (technology used for media transmission in modern browsers) I found so far is Erizo. I may have to implement an Erizo API module for Python if I decide to stick to Python.
So that’s the only issue I can somehow overcome. I cannot fix the internet speeds, though…
If I have enough time and knowledge, I’ll implement:
- voting system
It’s going to be exciting summer. I’m looking forward to it! :)
I welcome and highly appreciate any feedback, ideas, suggestions, or complaints. I released publicly my proposal, so you can comment it or even fork and make a better one.
I also want to share the link to the Git repository, where my work will reside. Feel free to send pull requests :)
On Wednesday 28 Aug, 2013
Due to my efforts, Ganeti Web Manager is now a proper Python Package.
The main reasons for this change are:
- minimizing necessary dependencies (no more fabric!)
- keeping GWM codebase clean
- easing development
What does this change mean?
For end users
With GWM being Python package comes one huge advantage: easier installation. It’s actually one line to install whole GWM with dependencies!
Because my GSoC project is to make GWM installation easier for end users, I’m writing a setup script, that will create virtual environment (helps with separation), install GWM’s dependencies and then GWM itself.
To install Ganeti Web Manager, simply get setup.sh script, make it executable and run setup.sh -h to get familiar with it’s command line arguments.
$ wget -c https://github.com/pbanaszkiewicz/ganeti_webmgr-setup/blob/develop/setup.sh $ chmod +x setup.sh $ ./setup.sh -h $ ./setup.sh -d /opt/gwm
As of 29/08/2013, this setup script is Work In Progress (TM). Some parts of it may not be completed and simply won’t work. Be careful. Or wait for Ganeti Web Manager next release, which should be 0.11.
To start working on GWM, you have to:
- make virtual environment
- git clone GWM repository
- install GWM as a development package
(I suggest using virtualenvwrapper for #1, as it keeps your directory with code clean.)
In shell terms it looks like this:
$ mkvirtualenv gwm (gwm)$ git clone git://git.osuosl.org/gitolite/ganeti/ganeti_webmgr (gwm)$ cd ganeti_webmgr (gwm)$ python setup.py develop
And that’s it, now you can work on GWM. (Alternatively you could install GWM development dependencies instead of whole package:)
(gwm)$ pip install -r requirements/development.txt
On Monday 08 Jul, 2013
My job was to refactor Ganeti Web Manager so that instead of one huge, bloated Django application (ganeti_web), the project would consist of many smaller apps.
I shared more details in this blog post.
Now I can say my work is mostly done. I’ve got smaller apps extracted and (this one’s big) I worked on database migrations.
Now (hopefully!) any user can checkout my branch and successfully run it against their codebase.
This work brings hope and sunlight to all the GWM developers. It helps new ones “see” and understand code, as the code is now simpler! And it helps old devs navigate through the code, while fixing or enhancing it.
I hope to get my work merged soon, although it’s quite big change so the proper code review is really necessary.