GSoC '13 is over!

Well, Summer of Code 2013 is officially over. My project has pretty much met what was in my proposal - the stats and ranking patch is r+ (though awaiting Florian's approval) and should land soon.

There are a few leftover bugs that I'll be working on when I have time (I've been pretty busy with my coursework lately). Many of them are small (and possibly quick to fix): bugs 2046, 2048, 2069, 2081, 2116, 2076, 2145, 2146, 2184, 2185.

Here's a broad overview of my work over the summer:

• Modified the tabbrowser to accept generic tabs (bug 426).
• As an example usage of the new tabbrowser changes, made it possible to open about:* pages in a tab (bug 2002).
• Wrote the initial new conversation tab, with a list of contacts and a filter box (bug 2015).
• Added support for existing conversations in the new conversation tab and moved the management of possible conversation data to a new ConversationStatsService (bug 2055).
• Wrote an API to request protocol plugins for chat room lists, and implemented it for JS-IRC. Made the stats service use this API to display IRC channels in the new conversation tab (bug 2066).
• Experimented with IndexedDb to store conversation stats. Dropped it in favor of a JSON cache until we migrate to a more sophisticated database from which we can query possible conversations on the fly (and possibly incorporate log indexing?). Wrote code to store these conversation statistics and use them to sort possible conversations. Adapted Florian's log sweeper code to parse JSON logs for stats (bug 2143).

I've learned a lot over the summer. That includes technical stuff - JavaScript, XUL and the Mozilla platform, XPCOM, etc. - as well as a much deeper insight into open source development. I'd contributed to open source before GSoC, but mostly just small patches. This summer I was actively involved in the development process, and got to see how much care and attention is paid to detail when making even small changes. I had to consider things I would've ignored before - localization, accessibility, edge cases, UI profiling, etc. Every UX decision was preceded by lengthy brainstorming and debating sessions on IRC. It wasn't uncommon to spend 10 minutes (sometimes more) figuring out the best name for a variable!

A huge thanks to my mentor Benedikt Pfeifer, and the Instantbird team for all their help over the past three to four months, they've been a pleasure to work with! Be on the lookout for Instantbird 1.5, it's going to be pretty awesome (:P).

PS
You can download a change log of my project from here, or if you have the source, you can generate one yourself using
$hg export `hg log -b default -u nhnt11 | grep changeset | cut -f3 -d ":"`

Progress report: September 6th

This post has been long overdue. Here's the major progress over the last couple weeks:

• Bug 2066 is merged! Awesometab now queries accounts for chat room lists and displays them (only implemented for IRC, you'll have to flip the chat.irc.allowList preference to true in about:config).
• Bug 2143 - Ranking and stats are working and have had a couple of review cycles. We decided that efficient and intelligent searching/filtering is a requirement and also that the log sweeping code could be reused for log indexing. Based on all this the IndexedDB has been dropped from this bug in favor of a simple JSON cache for now.

First thing I'll be doing next is optimizing the chat room code from bug 2066 to eliminate UI lag while receiving chat rooms. After that, the priority will be landing bug 2143. Things will likely get slow after that while I have my first test cycle, but once that's over (or possibly before, if 2143 lands quickly) I'll be looking at optimization and better searching (implement a proper database, indexing, etc).

Cheers!

Weekly report: August 21st 2013

During the last 7 days, I didn't quite get around to everything I wanted to do (which wasn't helped by 4 of the days being holidays) but there was progress.

• Bug 2066 (chat rooms and IRC's LIST) is finally r+.
• I experimented more with frecency scores ("frecency" is inspired by Firefox's Places Frecency algorithm)
• After a bit of experimenting, I finally decided to stop messing around and get some actual data to play with from logs.
• Using Florian's WIP from bug 1584, I put together an addon that goes through logs and obtains information on recency (date of most recent log file) and frequency (number of messages exchanged, with separate counters for incoming and outgoing)
• Playing with the algorithm for a while, I finally ended up with the following:
• Score = Total message count * frequency multiplier * recency multiplier
• Here, frequency multiplier is the ratio of outgoing to incoming message counts, and recency multiplier is identical to what Firefox uses.
• This gave me pretty accurate results, the conversations with the highest scores were definitely what I would most likely want to open if I pressed Command+T :)

Tomorrow I will be working to integrate this into the stats service and use these scores to sort conversations in the newtab.

PS.
I again notice that my work over a week seems to be concentrated over one or two days, during which I spend 10-15 hours each day on a kind of frenzy. Meh.

3-day report: August 9th, 10th and 12th 2013

• Played with ranking - using frequency as well as recency as parameters (took most of my time on Friday)
• Made changes to the way ranks are stored, to better accommodate chats (soon).
• Updated tabbrowser images for retina displays (bug 2103) (Monday night)

Daily report: August 8th 2013

• Rewrote the way conversations are stored in IndexedDB. This took most of my time tonight (pesky bugs!).
• Now, a conversation's rank is stored with the conversation's id as the key.
• I've defined a conversation's id as the string obtained by concatenating its source and _id values.
• For contacts this looks something like "contact2", "contact-5", etc. For chats, it's source + accountId + name - for e.g. "chataccount1#instantbird".
• Added basic ranking. A conversation's rank is the number of times it has been opened. Ranks are updated and stored as conversations are opened.
• Short term to-do list:
• Code cleanup
• Support for chats
• More sophisticated ranking
• Obtain historical data from logs

Weekly report: August 7th 2013

• More modifications to the chat room list patch (bug 2066) - unfortunately this still isn't ready to land yet.
• Brainstormed the ranking system and storage of conversation data.
• Read documentation on IndexedDB and wrote code to store/retrieve contacts in an IndexedDB object store (mostly experimenting with the API and understanding the approach).

Daily report: August 1st 2013

Today I finally discovered what was causing performance issues I was trying to debug for a couple of days. Lag was being caused by tons of warnings being spewed due to bug 2075.

Having fixed this I was finally able to finish a new patch and upload it for review.

Weekly report: August 1st 2013

• Worked further on bug 2066. Hopefully this should be ready soon.
• Submitted patches for follow up bugs: 2068, 2075, 2076, 2078, 2081, 2082.
• Progress has been a bit slow. I need to speed up and get the chat room stuff done so that I can start working on statistics and ranking.

Daily report: July 24th 2013

• Further work on bug 2066 (WIP attached).
• Brainstormed the best way to cover up the awkward pause between clicking a chat in awesometab and the new tab showing up. For now, made the IRC joinChat method create a new conversation and then send the JOIN message to the server.
• Various improvements, see commits on BitBucket.
• Created patches for followup bugs 2051, 2065. New patch on bug 2055 as well, addressing feedback comments.

Daily report: July 23rd 2013

I've mostly been working on adding functionality to get lists of chat rooms for accounts while the patch for bug 2055 awaits review - in particular IRC's LIST. Here's a progress report:

• Added the necessary additions to imIAccount.idl (an interface that represents chat rooms, and a method in prplIAccount that returns a list of rooms).
• Added implementations for these additions in jsProtoHelper.
• Consumers can now ask IRC accounts for LIST data, which will be returned asynchronously via a callback.
• Created an implementation of PossibleConversation - PossibleChat - to use with these chat rooms. These are now filtered and returned in getFilteredConvs, along with contacts. New conversation tabs can use these to create list elements and open conversations.

To-do:

• Proper data management for the chat room lists that stats service receives.
• Performance improvements, which will likely be helped by:
• Add ability to return list data in batches (Freenode for example has way too many channels to return all of them at once).
• UI improvements for MUC items.

After this is done I will get to work on the statistics and ranking part of my project, which will likely be the main focus of the second half of the summer.

Weekly report: July 22nd 2013

• Made a lot of modifications to the patch for bug 2055 to address the feedback I got for the first one.
• There are now multiple instances of PossibleConversation (PossibleContact, ExistingConversation, PossibleChat to come). The contacts are maintained in one list (as of now) and UI convs are injected into it before filtering.
• Did a lot of refactoring in the newtab binding with clearer naming and future requirements in mind.
• Existing conversations (both already in a tab and on hold) are now displayed in newtabs.
• Fixed a few follow up bugs (bugs 2041, 2045, 2047, 2063).

This week's plan is to get started on the ranking system. First goal is to have LIST working from IRC, then storing the data in JSON (as of now, but I'm keeping the IndexedDB option open).

Daily report: July 16th 2013

• Today I added support for conversations on hold in Awesometab.
• This took most of my day. I spent more time than I'd have liked to while figuring out how the blist window manages convs on hold and how to extract the data I need from the [showing-]ui-conversation[-hidden] observer notifications.
• I've attached a patch on bug 2055 and requested feedback on it.
• I still need to make it display open conversations (but not on hold).
• I plan to do this and have a complete patch with comments ready tomorrow.

Weekly report: July 16th, 2013

The past week:

• The buddy list tab is merged
• Quite a few changes were done in order to address more review comments Florian before the merge.
• Lots of followup bugs have been filed (bugs 2041, 2043-9). I have WIPs for most of them, and patches submitted for a few.
• Did research on the account interfaces for adding methods to return MUC lists (and a bit of experimental code - I was able to open IRC channels from awesometab)

To-do next week:

• Display conversations on hold in awesometab.
• Add the new function(s) in the account interface to get MUC lists.
• Implement this in particular for IRC LIST.
• Display IRC channels in awesometab.
• Also while doing this, address followup bugs to bug 2015 (including the ones mentioned above)

Pseudo-weekly Report: July 10th 2013

I call this "pseudo-weekly" because it's been a week since my last report :(. I was afk for a few days at the end of last week (literally didn't open my laptop :( ) so progress has been a bit slow.

• More review iterations on the buddy list in a tab bug (but 2015)
• Created a new interface ("ibIPossibleConversation") that will encapsulate both contacts as well as non-contacts like MUCs and people on IRC who haven't been added to the buddy list.
• The stats service and newtab binding now work with PossibleConversations instead of imIContacts.

Daily report: July 1st-2nd 2013

Monday, July 1st:

• Mainly a bunch of UI tweaks. This took a while. The new fonts are much cleaner looking, and some icon alignment issues are fixed. Also included the aero version of tag.png.
• Fixed a few bugs relating to key events.

Tuesday, July 2nd:

•  Looked at the implementation of the LIST command on IRC to get some idea of what kind of data I would have to process for MUCs in Awesometab.
• Moved all of the data management code from awesometab.xml to a new service (ConvStatsService) that will in the future manage contact statistics and sorting for Awesometabs.
• Learned quite a bit about services and interfaces (particularly about writing IDL files!) on the way.
• The buddy list tab now uses the service and is fully functional.

Daily report: June 28th 2013

• Fixed some bugs related to observer notifications and the updateContact method.
• Updated my Pastebinner addon to allow the user to set a default language for pastes, and also detect diffs (startsWith("diff") ;) ). Nominated it to be public, and aleth approved the request.
• While doing that, found a bug - Services.cmd.unregisterCommand wasn't working. Filed it and aleth (after "half an hour of debugging"!) found the problem that was right under our noses.
• I went through the code and did a sort of self review - fixed some nits, added comments, did some general clean up. Found a couple of minor bugs and fixed them.
• Incorporated Mic's icon and Firefox's tag icon.
• Finally uploaded the new patch for review.

Daily report: June 27th 2013

• Did some more work on handling of DOM elements. Instead of permanently linking a list item with a contact and removing/adding them when resorting/filtering, now the required number of items is maintained in the list and each is refreshed with the correct contact for its index. I know this sentence may not make complete sense, the code in my repo might make it clearer. This was inspired by a suggestion from Mook on IRC.
• Items are now appended as the user scrolls.
• Performance is greatly improved. Very little to no lag while loading contacts, resorting, etc.
• Added Mic's favicon and also started using the tag icon from Firefox.
• Submitted a patch to add a method to the Contacts Service to return an array of all contacts. This is checked in.
• Started maintaining a short-term todo list in the etherpad.

Daily report: June 26th 2013

• The tabbrowser patch is finally checked in, as is a patch adding an /about command to show about:* pages in a new tab!
• Today I worked on rewriting the awesometab buddy list code to better handle DOM elements.
• I've now got it dynamically removing and adding elements as necessary rather than displaying the whole list and filtering with CSS. Code is uploaded to my repo.
• There was a discussion on executeSoon and avoiding multiple calls to a function in a short time on IRC.

I am going to try and write these logs from now on. They may be of some use later on, or just nice to look back at.

Weekly report: June 23rd 2013

This week the tabbrowser patch saw a lot of tweaking (four review iterations iirc) and is now almost checkin-ready. I also worked on bug 2002 - adding a command to display about: pages in a new tab. This was a good demonstration of the new tabbrowser features and might help iron out any remaining bugs once it lands in the nightlies and users try it out.

I also worked on the awesometab buddy list. There were discussions about the UX on IRC and I experimented with a few UX ideas (async loading of contacts, alternating background for list elements, displaying only a few contacts at a time and loading more while scrolling, etc). Many of the to-dos have now been crossed out (see previous post). Next week I hope to report much more progress on the UX and possibly some initial details of the filtering/ranking system that is a major part of my project.

A few things that I want to mention:

• Mic created a cool icon for Awesometab. Thanks!
• I learned more about the working about XBL. I had a few frustrating moments because I did not realize that bindings were applied only after adding the element to the DOM (because they are applied from CSS) until Mook kindly pointed this out on IRC (thanks for saving me a lot of time when nobody else was around!). This same issue forced me to be extra careful while dealing with addition/removal of contacts from the Awesometab buddy list.
• I still haven't received my GSoC welcome package and hope it comes soon :(

Weekly report: June 15th 2013

It appears I can't remember to post daily logs. Instead, I've decided to post whenever something stumps me, or I discover something interesting, or in general when I want to log something I run into.
Now that that's out of the way, here's what I've done this week:

• The tabbrowser changes are now pretty much complete and are awaiting review! There are still some nits and other things I need to address, but it should be pretty minor stuff.
• Tabbrowser being tamed, I set out to start work on Awesometab. The goal is to have a buddy-list type UI as an initial milestone. So far, here's what I've managed:
• Awesometab can be launched via keyboard shortcut (Ctrl+t), context menu, and the new tab "+" button.
• It displays a list of buddies - nothing fancy, just a list of names - and a search bar.
• On typing into the search bar, buddies are filtered based on whether their display name starts with what has been typed (not case sensitive).
• When the "Enter" key is pressed, conversations are started with all the buddies selected in the list, and if no buddy is selected, the first one in the list.
• I'm pretty happy with my progress so far, but a lot remains to be done! Here's a to-do of what I want to finish in the coming days:
• Opening an Awesometab in a new window needs to maintain the current value in the search box and currently visible buddies.
• Adding buddies to the list should be async.
• Show conversations on hold in the list.
• Make things pretty. The buddies should be displayed with their profile pictures, status messages, availability, etc.
• Sorting - based first on availability, then alphabetically. Protocol must also be taken into account. Right now the list is completely unsorted.
• A bit of polish - double clicking a buddy in the list should open a conversation, a context menu, etc. - basically adding in more buddy list functions.

With that I sign off! Have a great weekend!

Daily report: June 9th 2013

• I fixed tooltips for generic tabs.
• I merged the latest default branch into my awesometab branches. (I should do this more often)
• I filed a bug and submitted a patch to fix indentation in buddytooltip.xml.
• I reviewed my changes so far, made a couple corrections and uploaded a patch requesting feedback. I plan to update the patch with more comments and the buddytooltip.xml indentation fixes before requesting review.

Weekly report: June 8th 2013

Most of the tabbrowser cleanup is now done! Here is a commit merging these changes into the awesometab branch in my repo. I've been working on these changes the past two to three days - though I had some personal things to take care of which took some of my time.

To summarize the tabbrowser work so far:

• Tabbrowser-tabs are now fully transparent to the nature of their linked panels. The tab panel a tab is associated with is now stored in its linkedTabPanel property, though the linkedConversation property is still set (identical to linkedTabPanel) - it can be used to check if a tab holds a conversation, and conversation specific code is more easily understood.
• Tabbrowser now accepts non-conversation tabs and displays them without errors.
• All basic functions like moving tabs around, detaching and reattaching, opening in a new window, etc. work for these tabs.
• Tab specific context menu items are handled by getting a NodeList containing menuitems from the tab panel.
• Cleanup has been done to make tabbrowser as transparent to tab content as possible.
• Fields like mCurrentConversation, mCurrentBrowser, etc are replaced by properties which return them from mCurrentTab. This saves setting them in different places - now only mCurrentTab needs to be set.
• Some changes have been made in other files to make them aware that not every tab holds a conversation (keyboard shortcut commands for e.g.).

What's left:

• Tooltips need to be taken care of. I've looked into this and it doesn't seem to be too tricky. I currently plan to do something similar to the menuitems, where the tab panel returns its tooltip data.
• It needs to be decided whether the tabPanel binding - which now functions as an abstract class - should be kept or not. Though it doesn't do much, it can be good documentation for developers who want to add non-conversation tabs.
• As an alternative to having pre-defined methods that are expected to exist in all tab panels, DOM events could be fired and tab panels could respond to them as they wish (as suggested by flo).
• After this, the changes need to be reviewed and checked-in to fix bug 426.
• This isn't exactly a pending task, but it was proposed in IRC that awesometab's algorithm could learn from logs so that users could benefit right away.

Daily report: June 5th 2013

My tabbrowser work has now reached a stage where a tab linked to an arbitrary panel can be added to a conversation window, moved around, etc with almost no errors. Context menus and nearly everything else also work fine. I only ran into one pesky error with the zoom manager, caused when the tab is opened in a new window - this.mCurrentBrowser is undefined because the tab doesn't have one. Debugging that will be a job for tomorrow!

The code has been pushed to the awesometab-experimental branch on my repo. Also I was a bit bored and recorded a demonstration video - here.

Meanwhile, I got default prefs working with my Pastebinner add-on thanks to the good folks at #instantbird.

To-do:
- Fix the bug mentioned above.
- A lot of cleanup has to be done so that the code is worthy of a check-in.
- Mic suggested I start collecting data to test my sorting algorithm later on - I want to write an add-on for this.
- I need to submit my add-ons to AIO.

Daily report: June 3rd 2013

I forgot to write up yesterday's report, so this post will cover what I've done the past two days.

• I decided to scrap the idea of having a generic tab to be extended, since tabbrowser-tab has very little conversation-specific code anyway. Now, tabs themselves are fully transparent and there is no need for them to be extended.
• The conversation binding now extends tabPanel - a generic panel to be added to tabbrowser. Future UIs that want to open in a tab should also extend tabPanel.
• I moved conversation specific menuitems from tabbrowser to the conversation binding. These are now "injected" into the context menu when it is shown.
• Code for the above has been pushed (except for the last bit which is awaiting review) to my repo.
• Aside from this, I worked on an addon - Pastebinner - which pushes any message more than 10 lines long to a pastebin and sends the link as a message. It also adds a "/paste" command using which files can be pastebin'd and sent. I suspect this will be useful throughout GSoC.

Daily report: June 1st 2013

I wasn't at home most of the day, but I did get some time to work after dinner.

• I was able to start over work on moving conversation specific code out of tabbrowser.
• Here's a diff of progress so far.
• Here is a discussion on this on #instantbird.

Daily report: May 31st 2013

Today I experimented with the abstraction code, ending up with more breakage than functionality.

• I worked on replacing linkedConversation with linkedPanel, since linkedConversation is mostly used in a way that doesn't depend on it actually being a conversation.
• I moved the tab context menu code to the tabbrowser-tab binding. This worked, but didn't yield very good results. Tab images got a bit broken and there were some mouse event issues.  I decided to move it back to tabbrowser, with the idea that generic tabs which wanted to implement their own context menus could do so by setting unsetting mousethrough="always" - this would show the tab's particular context menu rather than tabbrowser's.
• I added in a aTabClassName parameter to addGenericTab to allow tab classes other than tabbrowser-tab.

Progress was slow again because I had to decipher a lot of vague/complicated code (e.g. this.parentNode.parentNode.parentNode.parentNode.foo(this)) and wasn't able to open DOM Inspector until Mook_as mentioned the -inspector command line flag (thanks!).

To-do: make tabbrowser-tab easily extendable and further work on tabtest.xml to be a full proof-of-concept for adding non-conversation tabs.

Daily report: May 30th 2013

With help from #instantbird, I was able to fix yesterday's bug where Mic was able to open an arbitrary UI in a tab but I wasn't (with the same code). I had to add the newly created tabtest.xml and tabtest.css to jar.mn. Then everything worked and I was able to open the test UI, the add-on interface, the account manager, etc. in a tab (screenshot). Of course, it was still pretty broken and there were a load of errors from different methods assuming that the tab held a conversation.

• Mic had a much better way of generating new tabs: create a new command ("/newtab") instead of clicking the buddy list (new bootstrap.js).
• I did some research into how best to eliminate the errors and add abstraction. The discussion included moving some of the code in tabbrowser to the tabs themselves.
• I somehow broke my local tree and decided to do a fresh clone.

Today was slow but I hope to make more progress tomorrow.

Daily report: May 29th 2013

Today GSoC really started to kick in. I started a repo for the project here. As Mic suggested, I started working on getting the tabbrowser to accept tabs that aren't linked to conversations.

• As a start, I made a clone of the _addConversation method in tabbrowser.xml, and made some changes to let it add non-conversation content. Here's a diff of _addConversation and addGenericTab with Mic's comments.
• I then created tabtest.xml and tabtest.css (for binding) in order to test it out.
• I created a small add-on (bootstrap.js) to add a new tab with the contents of tabtest.xml when the buddy list is clicked. I couldn't get it to work however.
• Mic got it to work from the error console using the same code, but we couldn't figure out what was wrong on my end and decided to debug further tomorrow.

Bootstrap

Hello, world!

This blog is dedicated to documenting my progress through Google Summer of Code 2013 with Instantbird - a Mozilla based instant messaging client. I will be working on Awesometab - a replacement for the current "Join chat" dialog that aims to be good-looking, efficient, and feature full auto-completion goodness.

My mentor for this project will be Benedikt Pfeifer (Mic on IRC). You can read my proposal's abstract here.

I'll be posting daily updates here detailing what I've been up to during the day, as well as posts summarizing progress on a weekly basis.

Here's looking forward to a great summer!