Blog by Piotr Banaszkiewicz

AMY release v1.4

Intense month of January is almost behind us, so it’s time for new AMY release.

Three versions have been released since v1.3: v1.3.1, v1.3.2 and finally v1.4.

Graduation

First, let me brag a little: on January 22 I graduated from my university, and have the Polish official profession title of engineer. You can imagine I’m like this all the time:

I'm engineer! All the time. (click to enlarge)

Release v1.3.1

One bugfix: don’t break whole timeline widget when there are TODOs without due date.

Release v1.3.2

New feature: stop using dots (.) for usernames, use underscores (_) instead.

This was an interesting issue: since we rely on some Ruby software on the SWC website, we can’t have dots in filenames (they’re treated as parameter access operator, for example: banaszkiewicz.piotr is piotr parameter on banaszkiewicz object). But we have filenames that correspond to usernames in AMY. So it was necessary to drop dots and switch to underscores…

Unfortunately, due to the way we have our project laid out on GitHub, some of the features implemented for v1.4 before this feature were included in the deployment; I will still put them to v1.4 section, though.

Release v1.4

The biggest highlights of this month are definitely:

  • first approach to the new API
  • API reports
  • merging events.

There were also some essential features, but not much. In v1.5 there will be a lot more.

Data fixing

We had to programmatically fix/complete some of our records:

  • historical events on production server were assigned an administering organization (that’s the one responsible for taking care of the workshop bureaucracy),
  • new DC instructors were added: anyone with a special note or anyone who taught at DC workshop now has a DC instructor badge.

Bug fixes

Looking at the list of issues for this release, it seems like many bugs were fixed. It’s true, however the bugs themselves weren’t that big:

  • some fields containing numerical values were switched to other type of fields to prevent slider from appearing; the background for this issue was that when scrolling through a page with form, on MacOSX people would accidentally change values of numerical fields,
  • generation of initial revisions was added to the process of creating a fake database for development use,
  • some types of events (stalled and unresponsive) were kicked out from debrief lookup,
  • some invoice options were changed to remain consistent with the rest.

New features

As usual, we hit a fair number of new features for AMY:

  • Person model is now able to store person’s occupation and ORCID code,
  • events can hold links to survey results (pre-workshop for learners and for instructors, post-workshop for learners and for instructors, and long-term for learners),
  • API call for getting members list is now for logged in users only, and returns members’ usernames too,
  • merging events: with option to select fields from either of events, or (in some cases) even to combine them together. The underlying code may be reused to fix persons merging,
  • workshop issues page now allows to filter workshops by assigned admin
  • move most of reports to the API; 3 reports now present a graph for easy use, 1 report was requested to be moved to API, and 1 new report was requested (and I made it in API),
  • API: new structure. It’s using hyperlinks between resources and allows to view and filter for example people associated with specific events,
  • slow tests were fixed (we gained probably around 10s on whole test suite, even though about 10-20 new tests were introduced); now it’s time to speed up migrations,
  • Greg added two new badges to the database: maintainer and trainer; I made sure to allow for editing badges via Django Admin interface, and also added these new badges to the fake database command,
  • Greg also added a new command for getting list of people who should be warned because their instructor training was about to close,
  • meanwhile I added a command for displaying report about instructor training completion rates.

Next release

I want to thank prof. Ethan White for his support to AMY development in January.

The next release may be last one made on such regular basis. The reason for this is that in March I start a new academic year (Masters!) and I know it will be very hard; what I don’t know is if I have time to work on AMY this much as in previous months.

Therefore there are multiple important features we want to implement in the v1.5 release – look for the “essential” issues.

AMY release v1.3

Welcome to 2016!

In the past month we’ve seen two releases of AMY: v1.2.1 and v1.3. This blog post contains a joined release notes for both of them.

Bug fixes

  • wrong URL used in event validation or import/update features is now indicated (and we won’t receive wrong notifications about it)
  • properly throw 404 on some pages (previously: 500)
  • spaces are striped from Person and ProfileUpdateRequest fields (names, emails)
  • disable location inputs on event details page if online country was preselected

New features

  • use custom-built jQuery-UI (so that we no longer have conflicts with Bootstrap’s tooltip module)
  • Greg updated the script used to send instructors “Hey, update your info” mails (it’s getting removed later on)
  • it’s possible to add memberships per host
  • new badge: DC instructor
  • new logic for dealing with two instructor badges
  • timeline of TO-DO items
  • basic models (e.g. lessons, tags, academic levels, etc.) are now manageable from Django’s admin interface
  • all persons view: add filtering by workshop type person taught at
  • remove blurred production database in favor of generated fake database
  • mailing script turned into better Django management command
  • bulk upload now shows generated username and suggested people with matching names
  • show preview of event on SWC website
  • API: filter events by tags

No longer with us

  • Greg removed some unused scripts (test-command-line-upload.sh) and commands (parse-instructor-info.py)
  • notifications about invalid HTTP header Host
  • other removed scripts and commands

Next release

January and February don’t seem busy for me, so I hope to have more done on AMY in the coming months. I also want to thank prof. Ethan White for:

  • continuing his support of my work through December
  • extending his support for the next two months!!!

See what’s scheduled for v1.4: https://github.com/swcarpentry/amy/milestones/v1.4

Automation Control and Robotics

I’m studying Automation Control and Robotics, a major that doesn’t say clearly what a person would do after graduating it.

From time to time I talk with people who either think I’m studying programming, or that I’m going to build robots in the future.

What is Automation Control and Robotics?

Most people don’t know what automation control is, so they focus on the part that sounds familiar: robotics. They automatically assume I’ll be building robots like these, or these.

Well, I won’t.

My studies concentrate on things like control systems (think of it as SCADA), optimization methods, control theory, electronics, specialized electronics (FPGAs, embedded systems, assembler programming, industrial-class robots), leverage of stochastic methods in industrial process identification, computer vision, PLC programming, and others.

As you can see, I was mostly balancing between engineering and very specialized computer science. There really was very little robotics during my Bachelor’s studies.

What can you do after graduating?

I like to call it: engineering.

People graduating automatics control and robotics are vast-minded, and ready to work in pretty much any engineering field that has something to do with programming:

  • we can set up wind-turbines
  • or air control systems
  • or nitrogen refill systems
  • or fine-tuning of power plants
  • or building assembly lines

There are thousands of options, all different kinds of industries.

Do I enjoy these studies?

Contrary to many of my friends, I do enjoy studying automation control and robotics. I learned a lot of engineering- and maths-related subjects, and I have hopes for a great work in future.

AMY release v1.2

This is the third post-gsoc release of AMY, the workshop management tool for Software-Carpentry. This release was supported by prof. Ethan White. Thanks a lot!

Below you can find release notes for version 1.2.

Bug fixes

This release contains bigger bug fixes than 1.1:

  • wrong object type passed to "".format() method
  • wrong characters permitted in event slugs
  • (big one) inconsistant logic in EventQuerySet.(un)published_events methods

New features

Following feature is also a bug fix, but it’s mostly a new feature so I included it here:

  • (big one) fix finding, parsing and validating of event tags: it will now work with <meta> tags on workshop websites

Other new features:

  • after approving person’s profile update request, the updated profile is displayed instead of the list of other update requests.
  • password reset form was added,
  • issues related to, for example, missing location data are now highlighted on event details page,
  • (big one) admins can now be assigned to specific events or event requests,
  • Greg changed descriptions section names on “instructor issues” page,
  • the same page was updated by me so that we can have pending and “gave-up-on” trainees listed,
  • the previous feature was introduced thanks to new ‘stalled’ tag,
  • our API gained some filtering (go to https://amy.software-carpentry.org/api/v1/events/published/, click “OPTIONS” and look at “query_params”; these can be added to the API calls, for example: https://amy.software-carpentry.org/api/v1/events/published/?host=123456789&administrator=987654321)
  • the same API gained a new endpoint used for generating list of current members of Software-Carpentry Foundation; this is in no way official list of members, but it can be used to help determine who’s eligible (credits for this one go to both Greg and me since I finished his pull request),
  • it’s now possible to search in events’ URL, contact, venue and address fields,
  • 2 new options for invoice status were added (not invoiced for historical reasons and not invoiced because of membership),
  • more places (workshop issues, and on each workshop without attendance data) to send “Give us attendance figures” emails, more people to send to,
  • profile update requests can now be edited by admins.

Next release

There’s number of issues scheduled for v1.3 release, and there will be others added to that list. The problem is that December contains:

  • end of my semester,
  • one huge exam, a couple of smaller tests,
  • Christmas,
  • deadline for my BEng. project and thesis.

So total time spent on AMY in December probably will be lower than what I did in November.

AMY release v1.1

This is the second post-gsoc release of AMY, the workshop management tool for Software-Carpentry. Below you can find a release notes for version 1.1.

Announcement

I’m very happy to announce that development of AMY versions 1.1, 1.2 and 1.3 is supported by prof. Ethan White. Thanks a lot!

v1.1

Bug fixes

Can the low number of bug fixes mean that AMY’s getting mature? I hope so!

  • Small typo in README.md fixed.
  • Logic for displaying uninvoiced events was fixed so that events marked as uninvoiced but without any admin fee do show up on the dashboard.
  • Fields layout on the profile update request details page was changed to avoid confusion.
  • Better wording in email request to workshop hosts for attendees figures.

New features

  • New switch for Open/Closed requests (both profile update requests and workshop requests). Admins will be able to look up dismissed or finished requests.
  • “Recently changed” log was moved to a separate view so that it doesn’t clutter the dashboard.
  • Some objects (events, hosts, persons, airports, tasks) are followed by django-reversion. Thanks to that fact we can now show who+when created any of these objects and who+when last modified it.
  • Each event marked as “uninvoiced” can easily be invoiced by the admin thanks to the new “Invoice” button on the event’s details page.
  • New report: people who have been through instructor training but didn’t finish.
  • Self-organized workshops can now have a “self-organized” host.
  • New report: show how often instructors teach.
  • Admins can mark events as completed and throw them out of the filter results.
  • Big: events can have multiple TODO items associated with them.

Next release

There’s already a big pile of things to do for the v1.2 release. I can guess that only a number of the issues will be fixed. It’s good, because otherwise there would be nothing for me to do for v1.3. :)

AMY release v1.0

That was a looong ride, but we’re finally here! AMY is mature enough to be given the v1.0 tag.

This will be a joined release notes to multiple releases:

  • v0.9.1
  • v0.9.2
  • v0.9.5
  • v0.9.6
  • v0.9.7
  • and v1.0.0

v0.9.1

  • Greg Wilson has worked on the “problems” page: switched its layout to 3-column, added better email messages, etc.
  • He also extended our API to return airport’s country.

v0.9.2

  • I hot-fixed an issue with creating M2M with intermediary model in model forms (basically impossible, we have to do this manually).

v0.9.5

  • This release adds workshop request forms and profile update request form to AMY.

v0.9.6

  • I hot-fixed a bug with settings: we should load more email settings from environmental variables in order to use emails from Django.

v0.9.7

  • I added a new country: ‘Online’ (for international / online events, like some Instructor Trainings).
  • The dashboard was changed to show not only upcoming, but also ongoing workshops (in the same column).
  • Greg added some helper commands to report inconsistencies in PDF certificates for instructors.
  • He also removed old command used for checking AMY’s database against SWC site’s archive. Site has moved to use AMY’s API, so the archive was gone and the command was made redundant.
  • I updated check.py script (used for checking workshop settings correctness) from workshop-template repository.
  • Greg cleared up titles of forms introduced in v0.9.5.
  • I split these forms’ templates and added SWC and DC logos to them.

v1.0.0

Bug fixes

  • Bulk-uploaded CSVs with entries that contained empty rows (for example: ,,,,,,) were treated as correct, but not anymore.
  • “Artificial” country European Union had been removed and was brought back since some hosts were using it.
  • API now returns website URLs to events instead of original URLs.

New features

  • We have a new database for development use (I need to switch to fixtures some day…).
  • I changed the way AMY handles event URLs:
    • admins cannot use GitHub repository URL for event’s URL
    • migration was used to change all URLs to website-URL format (if possible)
    • I also fixed import/update from URL feature to auto-select country if possible
    • …and to auto-use correct website URL instead of repository URL.
  • AMY auto-counts event attendees if provided with a list of attendees (for example via bulk-upload).
  • External links now open in new tabs.
  • Now there are more options to Event.invoice (for example with reason for lack of invoice).
  • New remove awards functionality.

Next release

I asked our admins what they want to see next in AMY and I received very nice feedback. You can see all requests in v1.1 tag on GitHub. It’s scheduled for the end of October (but you can expect some delays :) ).

AMY update #7

With a week of delay (3 weeks after v0.8 release), I finally released AMY v0.9 today.

The list of changes for this release doesn’t contain one big thing I was hoping for, but it’s still decent.

This is also the last release that (partly) took place during Google Summer of Code 2015.

Google Summer of Code 2015

I’d really like to thank everyone who helped me during this Summer:

  • Raniere Silva for keeping eye on the participants’ reminders and for so many friendly chats over these months
  • Greg Wilson for mentoring me and for lots of enjoyable meetings
  • W. Trevor King for introducing me to some very advanced git methods
  • my application’s users: Amy, Giacomo, Maneesha, Tracy and others, for providing me with excellent feedback.

This was incredible Summer; I learned a lot and had so much fun. That’s great to see that we’ll release AMY v1.0 soon as a wrap up of 4 or 5 months of work.

Bug fixes

Okay, back to the change log.

  • Some workshop URLs weren’t translated between repository and website versions. This is now fixed and every workshop with one of these links will automatically display the other link as well.
  • It happened that users put whole links into Host.domain (in the form https://domain.tld). This caused errors in many places where we wanted to display link to that Host. The issue is now fixed: users aren’t allowed to put protocols or trailing slashes into Host.domain.
  • Some management commands were broken due to the fact that we switched underlying methods to use API endpoints, but we didn’t change the commands themselves.

Feature(s)

This release was focused on getting permissions sorted out and adding read-only access to people from Software-Carpentry Foundation Steering Committee.

We had to cut out the another exciting feature: workshop requests, profile update requests. The Pull Request for this feature is very big and we’ll release it in v0.9.5 somewhere in the next week.

AMY update #6

After another two weeks, AMY v0.8 was released today.

The list of changes for this release is really small.

Bug fixes

  • Some workshops that didn’t provide a list of instructors or helpers were erroring out during Import from URL. That was fixed in a v0.7.1 in between release.
  • Some of location fields (address or venue) and contact field were too short for some events. Their length was bumped to 255 chars.

Feature(s)

This release was focused on integrating main Software-Carpentry website and AMY.

On the AMY side, I implemented a basic REST API (read-only).

Changes to the website are still WIP: this is a big project and I still don’t get “the whole image” clearly, so the development is slower.

Plans for v0.9

Nothing for now, except triaged issues.

AMY update #5

AMY v0.7 was released today. You’ll find list of changes below.

This release contains surprisingly few changes, and the development seemed slowed a little, but that’s not a bad thing: Greg’s moving to Europe with his family and I’m going from time to time on 2-day trips (Wrocław tomorrow)

Bug fixes

This time I start with bug fixes.

  • Check if event’s starting date is earlier than it’s ending date.
  • Ensure event’s administrative fee and event’s attendance are both non-negative numbers.

Theme problem

This release was themed “fixing host/site/organizer representation in the database”, and I claim success!

The change was rather significant and required me confirming ideas with Software-Carpentry administrators and Steering Committee.

Rename “Site” to “Host”
Previously event’s host was incorrectly named “site” (ie. location).
Rename “organizer” to “administrator”
Event.organizer field was mostly unused and no-one knew it’s real purpose. By changing it to “administrator” we now have a place for our administrators to mark events they’re working on.
Filter by administrators on the “All Events” page
We can filter by Host, we can now also by administrator.
More location fields for Event
Workshops that have a public website (most of our workshops does) contain standardized location data we didn’t previously collect. This was required by some other features.
contact field for Event
Additional field we didn’t collect in the past.

New features

Faster testing
I switched the Travis-CI server to use fast Linux Containers. Testing time dropped from 1.5min to 45s.
Enhanced filtering
It’s possible to select multiple countries and preferable gender on “Find Instructors” page.
Lookup instructors closest to the event’s location
From event’s details page administrators can quickly go to “Find Instructors” and search by latitude/longitude.
“Update from URL” functionality
This works in a similar way to “Import from URL” from v0.6.

Plans for v0.8

Greg wants v0.8 to store demands for Instructor Training in AMY.

AMY update #4

Today is the deadline and release day for AMY v0.6.

Here’s what’s new and what’s changed in this release.

In-between releases

I was forced to add some hotfixes in v0.5.1 and v0.5.2.

v0.5.1 included a fix for the name of one of our lessons (was ‘dc/spreadsheet’, but it should be ‘dc/spreadsheets’).

v0.5.2 included a fix for the round-off error that was crashing AMY. If you’re more interested on the story, read this commit message.

New features

Deletion of tasks
Tasks can now be deleted from their details page.
Eventbrite link
Event details page will show link to the Eventbrite’s event page instead of sole key ID.
Quick link for new airport page
Accessible from “+” dropdown menu.
Auto-fill end-date on event form
Setting a start-date for an event will set-up an end-date (+1 day) automatically (most of our workshops lasts for 2 days). The auto-fill won’t happen if user put something into end-date field.
2-column layout on person-details page
First column contains awards and tasks, while the second column contains knowledge domains and lessons person can teach.
Facelifted Find Instructors page
Full-width layout, filters in sidebar, no setting of “wanted” instructors. Using GET for filters application instead of POST as previously.
Ability to add tasks for person from their edit page
Now admins can not only assign awards to specific persons from their edit page, but also tasks.
Tabs on person edit page
Because of using 3 forms and 2 listings on the person edit page, the split was necessary. The first tab contains original person edit form, the second tab contains awards and award-form, the third one contains tasks and task-form.
Facelifted bulk-upload
Another switch to full-width layout. Confirmation page for bulk-upload will now show what happens with specific entries (“will be created” or “it’s already in the database” kind of thing).
Lessons sorted by default
Lessons will appear in alphabetical order in Find Instructors filter and also in “lessons” column on that same page.
Importing event from URL
Admins can pre-fill event create form with data from workshop page.
Improved “failed to delete” page
Thanks to one small discovery, a big and ugly chunk of code was replaced with almost a one-liner.

Fixes

Display of tasks
Tasks now show links to events and persons.
Eventbrite event ID name change
Previously it was ‘reg key’, now it’s clearer.
Better fields in airport form
“Airport name” instead of “Fullname”, “IATA code” with link in the “help_text” instead of “Iata”.
Better list of tasks on person-details page
These tasks now contain links to related events and clearly indicate roles a person has.
Searching for “firstname lastname”
Searching for “Piotr Banaszkiewicz” now works!
Enforced uniqueness for event slugs
Having two events with the same slug resulted in a crash, so we decided to enforce uniqueness for event slugs. This not only prevents the bug from hunting us, but also prevents admins from adding events with slugs that are already in the database.
Awards will prevent events from deletion
If there are some awards that point to specific event, that event will not be deleted.

Plans for v0.7

There are two things that I want to work on for v0.7 release.

  1. I want to add develop branch and set it as the main branch for the repo.
  2. I want to fix host/site/organizer representation in the database.

The rationale for 1. is that I had to make two minor releases to v0.5 when we already had features for v0.6 merged to the master. If we had only kept stable releases in the master (git-flow), I’d not have that problem.