February 08, 2010
Asa Dotzler -- all ears
Both ears are now quite clearly up.

February 08, 2010 10:07 PM
Chris Cooper -- Tackling the Release Engineering:Future queue
The Release Engineering:Future bugzilla component alternately inspires feelings of sadness, loathing, and contempt…and that’s just within the RelEng team!
I’m certain most developers first response to having their bug moved to the Future queue is, “Oh, look, my bug has fallen down a well.” Historically speaking, that may not be far from the truth.
What goes in Future?
Why does the Future component make people get punchy? For a long time, the Future component has lacked a decent description, so developers don’t know what it means when their bugs are moved there. Many have started hording their gigawatts in anticipation.
Bugzilla currently has the following small description of the Release Engineering:Future component:
“For longer term projects that have been agreed should be done, but have no immediate plans to so. These are not be part of the regular recurring triage. Advanced planning and placeholder goals for next quarter also go here.”
Despite the grammatical errors, this description is mostly accurate, but what does it actually mean:
- Triaged bugs with no immediate owners go here.
- Sadly, most enhancement bugs end up here unless they will make the core release process better, more streamlined, etc. quickly.
- Bugs that are blocked on longer term projects in other groups go here until there is something for RelEng to do.
- Advanced planning and placeholder goals for next quarter (or later) also go here. That’s pretty self-explanatory.
Remember: this description is for the Future component ONLY. The RelEng team continues to pick up and work on bugs that need to get done on a daily basis.
I’m thinking about how to improve this description, and will get the description updated in Bugzilla once I have achieved some rough consensus. In the meantime, I’ve posted this description of the Future component in the wiki as an ongoing reference.
The (Ever-Increasing) Numbers
At the time of writing, the Release Engineering:Future component has 343 bugs in it. This number has grown steadily over the past year despite having more release engineers on staff, and having made a great many improvements to our release automation and our continuous integration infrastructure.
Our turnaround time for bugs in the Future component is also not stellar. At our urging, the Mozilla Metrics team recently started setting up a dashboard to give us various statistics on our Bugzilla usage. Bugs don’t get fixed in the Future queue, so it’s hard to make truly accurate assessments here, but there are a lot of aging bugs in there. According to the numbers, fully 50% of the bugs in the Future component are older than 1 month and more than 25% are older than 6 months. How this compares to other teams or areas, I can’t say, but it certainly makes me empathize with developers who feel their Future-ed bugs have gone walkabout.
If it makes you feel better in a sadistic way, the majority of bugs filed by RelEng team members go directly into the Future queue. We’re not overly happy about it either.
I have a way? Is that better than a plan?
Things are getting done, though. For example, over the past month, the number of non-Future, release engineering bugs that are actively being worked on has gone from a low water mark of 130 up to over 200 today. For comparison, over the same period, the total number of bugs in the Future queue has slowly crept up from a low of 302 to 323. Mozilla is growing along so many axes that sometimes it feels like keeping the increase in the number of Future bugs to a linear relationship rather than exponential one is an accomplishment in itself.
So how do we stop the increase and start wrestling the Future component back under control?
The first step is triage. Starting this Thursday, February 11th, the RelEng team is going to have bi-weekly triage meetings to specifically prune down the Future queue.
As part of the triage, we’re going to be touching every bug and updating the whiteboard field with searchable tags. Our goal here is to make it easy to find classes of bugs in the Future queue so that duplicates and overlap can be easily eliminated, and fixes can be batched as much as possible.
We’ll be keeping a list of the tags we’re using in the wiki in case you want to follow along.
There are some classes of bugs that we won’t be able to eliminate (e.g. future goals), but hopefully within a few months, we’ll have the Future queue back under control.
It won’t be a quick fix, but it’s one we’re committed to.
February 08, 2010 09:22 PM
Eric Meyer -- Events and A Day, Belatedly
I’m a bad conference organizer.
Why? Because we opened the An Event Apart 2010 schedule for sales back in, um, flippin’ November, and I never mentioned it here. Cripes, I never even posted when we announced the lineup of cities. I could go through the great big long sob-story list of reasons why 2009 was really tough and blah blah blah, but when you get right down to it, I fell down on my job.
Okay. So. Time to correct that.
(deep breath)
Hey everyone, check it out: the complete tour schedule for An Event Apart 2010! Woohoooo!
- Seattle: April 5-7, 2010 (yes, three days; more on that anon)
- Boston: May 24-25, 2010
- Minneapolis: July 26-27, 2010
- Washington, DC: September 16-17, 2010
- San Diego: November 1-2, 2010
We’ve got a pretty killer lineup, if I do say so myself. You can get the mostly-complete list from our opening-of-sales announcement last November. It lists the people we had confirmed at the time; there have been a few additions since then. Check out your city of choice to see who’s going to be there! (But always remember that speaker lineups are subject to change: speakers are people too, and life has a way of interfering with schedules. I myself had to withdraw from An Event Apart Boston last year due to a family emergency.)
The price to register for these two-day, one-track Events is the same as it was in 2009, and there are educational and group discounts available for those who are interested.
But wait, I just said “two-day” when the first show of the year is clearly three days. What gives?
Seattle is the site of our first-ever A Day Apart, a full-day workshop that can be attended on its own or as part of a full three days of Event Apart ecstasy. And the inaugural Day Apart will be nothing less than a detailed plunge into HTML 5 and CSS3 with Jeremy Keith and Dan Cederholm. Jeremy handles the markup; Dan gets stylish. It’s going to be fantastic. I’m going to be in the back of the room for the whole day, soaking up as much as I can.
If you want to attend just the workshop, it’s $399 for the whole day if you buy an early bird ticket (available through March 5th). The price goes up $50 when early bird ends, and another $100 if you show up at the door. But I wouldn’t recommend that last, because I don’t think there will be any tickets available at the door. Again: if you show up unannounced on the day of the workshop and ask to buy a ticket, we will most likely have to turn you away, because I expect that there won’t be any seats available.
On the other hand, maybe you’d like to experience more than just one day of AEA goodness. Maybe you’d like to go whole hog and attend both the two-day Event Apart and the subsequent Day Apart, soaking up all the knowledge and enthusiasm and camaraderie that typifies An Event Apart. And who could blame you? If you do that, then the total early bird price for all three days is $1,190, whereas buying the event and workshop passes separately would total $1,294. That’s right: you actually get slightly more than $100 off the cost of the workshop if you attend all three days, over and above the early bird discount. (Or you can think of it as getting $100+ off the cost of the conference. We’re not fussy.)
As it happens, these three-day passes have proved quite popular. So if you want to get your hands on one of those—or on any Seattle tickets, whether one, two, or three days—I wouldn’t wait too long. Our internal analyses suggest that there will come a time, some time before the doors open on April 5th, that the ability to buy a ticket will cease to be. It may even pine for a fjord or two.
As for the four shows that come after Seattle, well, they’re looking pretty popular too.
I know I say this every year, but I’m really excited about what we’ve got planned for the year. Jeffrey and I constantly and (we hope) consistently strive to create an event that we ourselves want to attend, and that’s absolutely true of the shows and workshop we have planned in 2010. I can’t wait to hear what the speakers and attendees have to share. Hope to see you there!
February 08, 2010 03:29 PM
Postbox Team -- New and Improved Support for Mac OS X
Postbox now integrates with the Mac OS X technologies and applications you rely on most to get stuff done. Here’s what’s new:
Mac Address Book Support
Postbox provides read and write support for the Mac Address Book. Keep all of your contacts in one location and sync them with your iPhone or MobileMe service.
Send Meeting Invites with iCal
Apple’s iCal can now use Postbox for sending calendar notifications.
Search for Mail Using Spotlight
Postbox provides full support for Apple’s Spotlight to search through message bodies, message header information such as To: or From:, and attachment names.
Exchange Images with iPhoto
You can export photos directly from Postbox into iPhoto, and iPhoto will use Postbox to send photos as well!
Send Link from Safari
Easily send a web page link from Safari using Postbox. Very handy!
Lookup in Dictionary
Direct text-to-Dictionary lookups will help you quickly pick the perfect words.
Drag-to-Dock Message Creation
Dragging a file to the Postbox icon in the Dock will create a new message with the file attached.
Learn more about these new features and how to use them in our Quickstart Guide.
And now, with our new Refer-a-Friend Program, we’ve created an exciting way for you save on Postbox, or even get Postbox for FREE! See our Refer-a-Friend Program Page for details.
February 08, 2010 02:30 PM
Refer Your Friends, and Get Postbox for FREE!
Our Refer-a-Friend Program is easy! Tell your friends about Postbox, and earn $5 each time someone purchases with your coupon code! Refer six friends and Postbox is effectively FREE!
Step 1 - Save $10 on Postbox!
Use your search skills. Find a coupon code to purchase Postbox for just $39.95 $29.95!
Step 2 - Share your Coupon Code
We’ll give you your own coupon code for “$10 Off Postbox.” Share it via Facebook, Twitter, your blog, email… anywhere.
Step 3 - Make $5 each time someone uses your code!
You get it - make 6 referrals and we’ll send you $30 - the price you paid for Postbox. Any more… and it’s all gravy.
There you have it. Postbox for about the price of a pizza, plus a way to earn it all back and give your friends a discount. Everyone wins!
To get started, search Google or Twitter for a coupon code to Save $10 on Postbox today! If you are an existing customer, simply select License… from the Help menu, and click Get Referral Code.To learn more, please see our Refer-a-Friend Program page, read our FAQ, or review the Terms and Conditions.
A special “Thanks!” to Charlie Wood of Spanning Sync for his help and advice on the development of this program.
February 08, 2010 02:30 PM
February 07, 2010
Alex Vincent -- Asynchronous events and Mochitesting
Recently, I found that I needed my chrome Mochitest to wait for a XBL binding to apply, before continuing the test. For the end-user, waiting is hardly an issue - by the time they see the new user interface, all the bindings have applied. Mochitest (and most other automated test harnesses) runs straight through, and brakes for nobody. In some cases you can pull some trickery with nsIThreadManager, convincing Firefox to wait... but that's hardly reliable.
So I came up with a three-part solution.
- Force my XBL bindings to fire DOM events when the constructor executes.
- Break up my test functions into (slightly) smaller functions, each of which handles a part of the overall test. Two parts cover the main functional testing, while a third part is specific to the XBL binding.
- Implement a generic testing harness for Mochitest which can manage the flow from one function to another.
This new test harness, which I call PendingEventsTest, adds a layer of complexity on top of SimpleTest. Each test function is added via PendingEventsTest.addTest(), and you can pause for a DOM event with PendingEventsTest.addEventLock(). The whole test sequence can abort with an error by calling PendingEventsTest.abortFail(). You can advance to the next test function in the sequence with PendingEventsTest.asyncRun() or PendingEventsTest.run().
There's more complexity to it than that, of course. To move data from one test function to another, I also added "payload" support, for storing a generic object. Also, this is written for chrome Mochitests - it will likely fail in content Mochitests. Still, this new test harness helped me finish support in Verbosio for my equivalent of the <xbl:children/> element in markup templates.
Lastly, it (and the logging service I wrote a few weeks ago) is licensed under MPL 1.1/GPL 2.0/LGPL 2.1, so anyone who wants to borrow this for Mozilla code is welcome to. Cheers!
February 07, 2010 11:34 PM
Robert O'Callahan -- Reality Wins
On Friday night our family stayed with my parents on a boat up at Mahurangi, in Lagoon Bay. Easterlies had been blowing steadily for a few weeks and probably pushed a lot of plankton inshore, which helps explain what we saw that night.
About 10pm, after most of the crew had gone to bed and it was very dark, I noticed unusual flashes of light in the water. It was obviously bioluminescence caused by plankton, but far more intense than I've ever seen before. Even in still water, there were tiny points of light winking like flickering stars. Anywhere water was disturbed, it glowed brightly. Fish jumped, and the ripples formed bright, expanding halos. You could see moving, oscillating, sinuous lights underwater that I presume were fish swimming. I hopped in the dinghy with my wife and rowed around. The splashes of the oars created amazing flashes and patterns of light, and the dingy left a glowing wake. My wife dipped her hand in the water and it came out luminous.
It was an utterly remarkable experience --- like the bioluminescent forests of Avatar, but with the considerable advantage of being real. (Sorry, no photo --- the lights were only bright relative to the darkness of the night, and I lacked the technology and skill to capture them.)
February 07, 2010 08:25 PM
February 05, 2010
Vladimir Vukicevic -- mjs: Simple Vector and Matrix Math for JS
One common thread running through the many different and interesting WebGL projects out there is that they all need to do vector and matrix math, do it quickly, and do it in JavaScript. To date, developers have either rolled their own, or they've used Sylvester, a fairly featureful vector and matrix JavaScript library.
One of the problems with Sylvester is that while it's fully featured (arbitrary NxN matrices and vectors can be created and manipulated), it suffers in performance because of it. Since this is such a crucial part of a successful WebGL program, I've put together a small package that I'm calling mjs.
mjs is designed around speed and simplicity. For example, it doesn't attempt to stuff vectors and matrices into JavaScript objects. Because the language offers no operator overloading, there's very little benefit in treating these types as discrete objects, and lots of performance and memory usage downsides. Instead, it provides a set of functions for performing operations on vectors and matrices, which can be any array-like object. For any function that returns a vector or matrix, an existing array can be passed in to take the result, or the function can create a new one. Array reuse ends up being important because of the potential for expensive garbage collection churn eating away at performance.
Here's a sample of the API:
var r = M4x4.rotate(Math.PI/2, V3.$(0, 1, 0), M4x4.I);
Note that V3.$ and M4x4.$ are shorthand for creating a new V3 or M4x4 (I wanted to use V3() and M4x4(), but that didn't work out too well since functions have a length property). However, because all they return are just new array-like objects, you could also write:
var r = M4x4.rotate(Math.PI/2, [0, 1, 0], M4x4.I);
If the WebGL types are available, those will be used for newly created vectors/matrices. They are a significant performance boost especially for repeated operations; but for specifying one-off vectors such as the above, literal array syntax is fine.
The rotate function internally makes a rotation matrix, and then multiplies it by the given matrix. So the above could also be written as:
var rotation = M4x4.makeRotate(Math.PI/2, [0, 1, 0]); var r = M4x4.mul(M4x4.I, rotation);
(The last line being redundant given that we're multiplying by the identity matrix.)
All methods that return a vector or matrix take an optional final argument, that of an existing object to reuse. For example:
var m0 = M4x4.$(); r = M4x4.mul(someMatrixA, someMatrixB, m0); // r == m0, so the assignment isn't necessary, but it's handy for chaining // .... do something with r ... r = M4x4.mul(someMatrixB, someMatrixC, m0); // r == m0 still // ... do something else with new results ...
Without allocating any additional temporary objects.
As mentioned before, one of the goals of mjs is performance. Matrix multiplication is one of the most common tasks, so here are some numbers comparing mjs, Sylvester, and native C code. This was run on a Core i7 desktop using a local build of Spidermonkey, which included one patch that's about to go into the tree that fixes the no-reuse tracing case. (Without it, the no-reuse tracing case is much larger because it's never actually jitted.) The test is simple: it multiplies two matrices together in a loop 1,000,000 times.
| Test | Time |
|---|---|
| mjs, JIT, matrix reuse | 140ms |
| mjs, JIT, no reuse | 533ms |
| Sylvester, JIT, no reuse | 5,280ms |
| mjs, no JIT, matrix reuse | 25,833ms |
| mjs, no JIT, no reuse | 26,681ms |
| Sylvester, no JIT, no reuse | 41,996ms |
| Native C++, SSE2, matrix reuse | 71ms |
| Native C++, SSE2, no reuse | 142ms |
(I also have numbers for MSVC without the SSE2 compile flag, but the numbers vary greatly depending on whether the values eventually go to infinity or not; if the values end up trending towards 0, the non-SSE2 code tends to win at around 52ms vs. 71ms; if the values trend to infinity, the non-SSE2 code takes around 11,000ms!)
Those numbers are pretty encouraging -- having native code be only 2x as slow for something like this is pretty nice to see. Granted, this is only a very isolated test, and I'm sure there are some tricks to optimizing the native code case (it's currently just a fully unrolled set of multiplies and adds). The "no JIT" case is less nice, but I'm sure that our Jaegermonkey folks will be all over this testcase (right, guys?). In any case, ideally most WebGL rendering loops will be fully traced in Firefox, so it would be less of an issue.
mjs is still very much a work in progress; it's missing a test suite and a whole bunch of features. You can find it hosted at Google Code, at webgl-mjs. (Side note: I couldn't just call the project mjs because a project called mjs was abandoned on Sourceforget 5 years ago, and Google Code complained.) There's also some documentation, viewable online here.
Bugs and contributions welcome!
February 05, 2010 09:04 AM
February 04, 2010
Dan Mosedale -- Lanikai Alpha 1
Thanks to hard work by a whole bunch of folks, we shipped Lanikai Alpha 1 (the first development release of Thunderbird after 3.0) yesterday. More details about Lanikai can be found on the project page.
This release is a first in several notable ways:
* we're now requiring automated tests to land with most code changes
* the release cycle was much shorter than any development release in recent memory
* we're now having to do development and releases across both a development and a stability branch.
We've already learned a bunch of stuff from all the changes, and I expect that learning to continue for a little while before we're fully in a groove.
Thanks to everyone who has been part of making the release happen!
February 04, 2010 09:59 PM
Boris Zbarsky -- Understanding the numbers your profiler gives you
A common situation I run into is that I have some testcase, I profile it, then I optimize one of the things that looks slow a bit and reprofile. If I'm lucky, then the fraction of time it takes drops from B to A (A < B of course, with B standing for "before" and A for "after"). What does that mean for the overall time of the testcase?
Obviously, what it means depends on both A and B. If we assume that the time taken for the part that wasn't optimized hasn't changed, then the overall speedup is (1-A)/(1-B).
So B - A = 5% would mean a 2x speedup if B is 95% and A is 90%, and only a 1.05x speedup or so if B is 5% and A is 0%. If B is something like 53% and A something like 48%, then you get a 1.11x speedup.
All of which is to say that focusing on the hotspots is _really_ the way to go, if at all possible.
February 04, 2010 08:21 PM
Chris Cooper -- All about the RelEng sheriff, revisited
This is a follow-up to Ben’s blog post about the RelEng Sheriff back in October. His post described clearly what the RelEng Sheriff (and more generally, the RelEng team) can do to help developers.
Since we implemented the RelEng sheriff (or “buildduty” as it is more informally called) last spring, developers have been getting better about using the buildduty person as the first point of contact for RelEng issues. I’d like to implore people to continue doing so. There are a few exceptions, of course:
- It is outside of regular work hours and/or the designated buildduty person is not available; or,
- you are already working on a bug with a specific release engineer.
We recognize that different release engineers have different core competencies, and it may be tempting to go “straight to the source,” but please respect our process. The process is in place to limit (as much as possible) the interrupt-driven nature of release engineering work and to minimize the impact of those interrupts on ongoing projects. In many cases, the person with the expertise you need will still end up helping you (we’re all in the same IRC channels, after all), but the buildduty person is specifically committed to helping you, even if only as a thin scheduling proxy for another release engineer.
Ben’s post had a good list things that RelEng can help developers with. Based on actual requests over the past few months, here are a few other services we can offer:
- clone a virtual machine for one-off debugging/testing: Sometimes patches work on the try server but (for whatever reason) don’t once checked in. Sometimes tests fail or are intermittently orange. If you’re seeing issues like these after landing a patch, or need access to a specific platform for short-term work, the RelEng team can clone one of our reference machines and give you access (under certain circumstances).
- grab build/extra logs from a build machine (provided they exist). This is often useful if a try server build generates extra, non-standard output
In both the above cases, the best thing to do is talk to the person on buildduty and then file a bug.
Conspicuously missing from Ben’s post was a list of what RelEng can’t do for you. While we can usually point you in the right direction, there are some things that we explicitly cannot do:
- help you debug issues with your own local builds. We’re not (necessarily) build config experts, so unless you’re running with exactly the same configuration as our build machines and don’t have any code patches applied, you’d be better served asking the talented folk in #developers or posting to the newsgroups (m.d.builds or m.d.a.firefox)
- shepherd your patch through the landing process. Standard patch landing rules apply: *you* are expected to be around when you land patches. If builds/tests fail and you’re not around, your patch will likely be backed out.
Need help finding the RelEng sheriff? Here’s the RelEng sheriff schedule.
February 04, 2010 07:37 PM
Gervase Markham -- Protecting Germans III: Our Allies
This is the third in a series of blog posts (previous: 1, 2) on how Mozilla is using its trademarks to try and make sure everyone gets a genuine copy of our software, for free.
Mozilla has a form where people can report sites abusing our trademarks. And we are very grateful to people who do. Harvey Anderson, the Mozilla general counsel, has recently posted some statistics on what those sorts of reports have helped us achieve.
But we aren't the only people trying to deal with the problem of "subscription trap" sites. downloados.de is a private initiative in Germany educating internet users about the typical set-up of a subscription trap. It comes across like a typical trap site, but when the user makes the final click to send off their personal details and to obtain the software, the website then informs them that they were about to spend money on free software and that they should be more careful next time.
February 04, 2010 11:00 AM
Asa Dotzler -- sodapop is growing up so fast
Sodapop turns 9 weeks old tomorrow. The most notable change from a week ago is that one of her ears has started to stand up. Eventually (probably in a week or so) both will be standing tall. The next change, besides weight gain and better manners, will be when more of her silver hair comes in. You can see a bit of it around her eyes, but she's still mostly black and tan.
Sodapop at 8 weeks
Sodapop at 8 weeks 6 days
For those people paying attention (and anyone who might want to update my Wikipedia entry,) Sodapop the yorkie is the third child in our family behind Ptolemy the cat and Munch the bunny.
February 04, 2010 08:44 AM
February 03, 2010
Vladimir Vukicevic -- Android Progress: More Pixels Edition
It's been a while since I posted a progress update (or really any blog post, ahem), but porting Firefox/Fennec to Android is progressing at a good clip. After working out a few kinks (and setting the all-important "you're allowed to touch the network" permission), I just got our first page load:
Mouse events sort of work, toplevel windows sort of work, keyboard doesn't work yet but shouldn't be hard to hook up. This is running in an emulator at the moment for ease of debugging, but it's working just fine on physical hardware as well.
You'll note that this is the full Firefox interface, and not the Fennec/Firefox Mobile UI; we're testing with the full interface because it's significantly more complex than the mobile UI and stresses Gecko much more. So, if the full UI works, then Fennec should work fine as well. Given the interest in Android on netbook and tablet devices, an updated version of the full Firefox UI might find a home on some of these. Android has been pretty great to work with so far; it's a bit unusual platform for us due to its Java core, but with the NDK we're able to bridge things together without many problems.
We're still a ways to go before any kind of usable alpha release, but we're certainly one step closer. We'll also be able to accelerate our progress now that we have some of the basic scaffolding in place. I know I'm looking forward to running Fennec on my Droid, and there are tons of Android devices coming out that should be great platforms for Fennec.
February 03, 2010 12:29 AM
February 02, 2010
Asa Dotzler -- mozilla store needs you
The Mozilla Store needs your help! If you have a minute, please take this quick survey
February 02, 2010 10:17 PM
Gervase Markham -- Protecting Germans II: The Steps
This is the second in a series of blog posts (previous) on how Mozilla is using its trademarks to try and make sure everyone gets a genuine copy of our software, for free. I will continue to use Germany as an example.
When we discover a wilful trademark infringement, we normally take the following steps, in increasing order of seriousness:
- Contact search engines to ask for the site to be delisted.
We have a good relationship with the major search engines, and being delisted or having their advertising removed does a great deal to "cut off the air supply" of fraudulent sites.
- Report the site to consumer protection agencies such as computerbetrug.de, antiabzockenet.blogspot.com and boocompany.com.
- Send a cease and desist letter.
In most cases, sending a C&D; sorts out the problem. If it doesn't, we:
- File a DENIC domain dispute and request cancellation or transfer of any infringing domain names.
Where there is a domain name involved which infringes Mozillaâs trademarks, we can file a domain dispute alongside sending the C&D.; For German domains, this is very quick and cost-efficient (national stereotypes at work :-). Upon cancellation of a .de domain by the infringer, DENIC automatically transfers the domain to Mozilla. (For other TLDs, it's not so automatic.)
We have already obtained numerous domain names this way. Examples include "mozilla.de", "mozilla-europe.de", "mozillamessaging.de" and "mozilla.fr", as well as dozens of typosquatted domains.
Unfortunately, for subscription traps, a C&D; rarely has any effect. So we move on to:
- Obtain a preliminary injunction (PI).
In Germany, PIs are also quick and cheap. Typically, it takes 1 or 2 days to obtain a PI. The proceedings are ex-parte (you only need one side in court), the other side is not even notified of the application for a PI and there is no discovery of evidence. You make your case, and the judge decides.
PIs become effective upon service. If they are not complied with, they can be enforced through penalty payment proceedings. However, a quicker way to take the site down is by approaching the provider. Under German law, internet providers are liable for infringing content once they are made aware of the infringement. Thus, a letter to a provider, accompanied by a copy of the PI, usually leads to the immediate shutdown of the site. (Remember, the site has already been warned of proceedings by the C&D; this is not coming out of the blue.)
So far, Mozilla has needed to file 6 7 (another one this week) PIs, and all of them were successful. In two cases, the other side appealed, but the PI was confirmed on appeal.
February 02, 2010 09:43 AM
Community Size, and Bug Quality
I've just finished watching Diederik van Liere's fascinating presentation (Ogg video, 20 mins), linked to by Mark, about his work with bugzilla.mozilla.org data. Download the full slide deck - you'll need it to follow along with my comments, which should be of interest to the community-minded.
Slides 9 and 10, the community dynamics slides, are worrying. Diederik explains the decline in the size of the community as "increased product maturity", but I'm not sure that's true. If it were, we would expect to see the number of people entering the community decreasing. However, that number has stayed roughly constant, with peaks surrounding a release as we reach out to beta testers, and as new users report bugs. It's the number of people leaving which seems to be increasing. Fewer people are hanging around once they get here. And I don't know why.
Slide 11, about what the final resolutions ended up being of bugs filed in each year, is fascinating. It would be really interesting to see that graphed at a greater granularity.
We have made significant progress in reducing the percentage of duplicate bugs filed. People complain about Bugzilla's search, but the percentage was a flat 25% from 2002-2004, and it's now a flat 13% - basically half what it was. Also, from a low of 34% of bugs filed being FIXED, we are now up at 56%.
There are no doubt lots of factors involved in this effect, including the reduction in the size of the community mentioned in the earlier slides. I'm really interested in working out what we might have done which has improved the situation. I have an utterly open mind on this. here are some significant events I can think of which may be relevant:
- 2000-10: duplicates.cgi created
- 2002-10: Bugzilla Helper created
- 2003-09: Specific search created
- 2004-12: Hendrix created
Can anyone else think of other events or software deployments which may be relevant? I'm happy to do the research to find out when they happened.
February 02, 2010 09:05 AM
L. David Baron -- Faster repainting in SVG foreignObject
In the past week, I landed a few performance improvements to code for SVG's foreignObject element. (To be exact, I fixed bug 403443, bug 418063, and bug 541188.) This was a small side project that resulted from trying to use foreignObject for my inverted tinderbox display and realizing that foreignObject (which was what I initially tried) was significantly slower than using SVG filters in HTML (which also turned out to be simpler). These improvements are all available in today's nightly builds.
In particular, when we need to repaint a small part of the contents of a foreignObject (for example, due to typing in a text field or hovering over a link), we now repaint only the part that we need to (just like we do when not inside a foreignObject), rather than repainting the whole foreignObject.
This is quite noticeable when interacting with pages, for example when moving the mouse over links or when selecting text in Mark Finkle's example (from his blog) or in Paul Rouget's demo (especially if you load http://www.mozilla.com/ and then hover over the menus at the top of the page).
February 02, 2010 05:00 AM
February 01, 2010
Gervase Markham -- MediaWikis Can Host Open Video: wiki.mozilla.org Enabled
The awesome Jeremy Orem has written a Mediawiki extension to allow the direct embedding of Open Video, and has turned this feature on on wiki.mozilla.org. Check out my demo page (you just type the normal HTML syntax into the wiki markup) and then feel free to use it to host Mozilla-relevant videos. (Note: wiki.mozilla.org has a 32MB file upload limit.)
February 01, 2010 04:26 PM
Nav4All Loses Data Licence, Shuts Down
In a follow-up to my previous post, here's an example of what can happen if your build your business on non-free data or software from a single supplier:
Nav4All navigation shut down by NavteqLetter to 27,625,631 Nav4All navigation customers
Dear Customers,
It is with the deepest regret that we hereby notify you that the global navigation of Nav4All and the Tracking & Tracing will go offline in 3 days. The reason for the same is that the data licence agreement with Navteq (a 100% Nokia subsidiary) was not extended, in a totally unexpected manner. ...
Nav4All worked on "hundreds of different mobile telephones of many makes such as Blackberry, Sony Ericsson, Samsung, Motorola, Android, HTC, Nokia, LG, Iphone, Ipod etc." Navteq is now owned by Nokia. It's not hard to join the dots. I'm somewhat surprised Nav4All call this "totally unexpected", though. It's the obvious thing for Nokia to do. And they bought Navteq over two years ago. If I were a Nav4All shareholder, I'd be having strong words with the management right now...
February 01, 2010 03:19 PM
iPad
Software freedom means never having to say "I'm very much hoping Apple continues to allow...".
And that's all you'll hear from me about the iPad :-)
February 01, 2010 09:33 AM
January 31, 2010
Alex Vincent -- The strangest crash I've seen in a while
Last week, I wrote about a new logging infrastructure I built to track down a bug. In the original post, I warned that logging wasn't a panacea - that it could cause more trouble than it solved. I was more right than I knew, as I added a bit of logging which led inexorably to a crash by JS_ASSERT failure. (For those who don't know what a JS_ASSERT is, it's an assertion inside the JavaScript Engine that all is well. Since JavaScript is one of the primary languages of the World Wide Web, these assertions are really important. If JavaScript is misbehaving, it can't be trusted. Fail one such assertion, and the program aborts, forcing the developer to take notice.)
At first, the crash didn't make a damned bit of sense. I prefer to write JavaScript, not just because I entered into software engineering (and Mozilla development) through JavaScript, but also because it's super-stable. In fact, the only new code I was introducing was in JavaScript. It shouldn't crash. Through an unlikely chain of events, it does.
This article details how I was able to analyze the crash itself, how logging set up the conditions for the crash, and how other logging ultimately proved that the crash resulted from a logging call. I've filed this as bug 543304. Ultimately, reducing this test case to a usable bug report will not be easy - but worth it. More details in the extended entry. (Warning: technobabble content ratio is high. If you do not want to see how this developer thinks and debugs Mozilla code, do not read on.)
January 31, 2010 07:33 AM
January 30, 2010
Mike Pinkerton -- DropBox
January 30, 2010 06:20 PM
Robert O'Callahan -- Ruapehu Redux
Five Mozilla people at the top of Mt Ruapehu 12 days ago. The crater lake is in the background.
January 30, 2010 11:29 AM
Three Lessons From A Pernicious Bug
We were having a lot of problems on Tinderbox with xpcshell tests randomly failing on Windows by returning exit code 1. Curiously, it seemed that almost any test could fail this way, with low probability, and the output log for each test indicated that all the actual subtest assertions passed, the process just mysteriously returned an exit code of 1 for no apparent reason.
Naturally I charged into battle with my sword +5 against orange (i.e., VMWare's record-and-replay debugging). The tests failed randomly about one run in a thousand, so capturing a failure wasn't hard. Then I verified that the problematic xpcshell run was passing 0 as the exit code for the final NtTerminateProcess system call, but the outer python script was indeed getting 1 from GetExitCodeProcess. Very mysterious!
I was at a loss for a while until some questions from Benjamin Smedberg led me to observe that in the failing run, the Google Breakpad helper thread was still alive at process termination, but in passing runs, that thread had exited before process termination.
Benjamin then looked into the Breakpad shutdown code and noticed it using TerminateThread to shut down its helper thread. The MSDN documentation for TerminateThread warns that that function is an extremely dangerous API, because it abruptly terminates the thread, leaving any shared resources it was manipulating in a possibly inconsistent state. Nevertheless, the Breakpad shutdown code seems to use it in a safe way ... mostly. We observed that that code was passing exit code 1 to TerminateThread. Changing that exit code to 44 made the exit code returned in our randomly failing tests change to 44. Clearly, somehow the exit code set for the Breakpad helper thread by TerminateThread was becoming the exit code for the process!
We think that, although it's not mentioned in MSDN, TerminateThread is actually asynchronous. The thread is not terminated immediately. If you're (un)lucky, the main thread can trigger process exit (passing exit code 0) before the helper thread's termination is complete. Then, somewhere in the netherworld of NT kernel process finalization, after the main thread has been terminated, termination of the helper thread finally completes and as the last thread to terminate, its exit code is recorded as the process exit code!
Moral of the story: never use TerminateThread. If you must use TerminateThread, you should probably try to wait for termination to finish (e.g. by calling WaitForSingleObject) before exiting the process, if you care about your process exit code. Another moral is that record-and-replay debugging rocks (but we already knew that). Yet another moral is that debugging closed-source operating systems sucks (but we already knew that too).
January 30, 2010 06:21 AM
January 29, 2010
Doron Rosenberg -- Google starts killing IE6 support
According to Google's Enterprise Blog, Google is starting to phase out IE6 support, starting with Google Docs and Google Sites by March 1st. While this won't affect the big enterprises (who probably don't use those Google services much), this is a good first step and as more follow IE6 in the enterprise will finally die.
January 29, 2010 10:26 PM
Boris Zbarsky -- Cross-compiling success
I took another stab at directly cross-compiling (not via distcc) using toolwhip. My previous attempt failed, but not due to libffi at all. The problem there was that I had the Mac ld in my path before the host ld. After fixing that and a few mozilla build system issues and making sure the build system can find the right SDK and the right ar and such I now have a cross-compile setup working. No make package yet, since that requires the dmg-creation utility which I don't have on the Linux machine. No --enable-breakpad support yet due to bug 543111. --enable-shark needs headers I don't have on that machine yet, so I'd need to sort that out. But for basic bisecting, this works.
Just so I can find it later, the working mozconfig file I'm using is something along these lines:
mk_add_options MOZ_MAKE_FLAGS="-s -j10" export MOZ_MAKE_FLAGS="-s -j10" mk_add_options CC="ccache /Developer/usr/bin/gcc-4.2" export CC="ccache /Developer/usr/bin/gcc-4.2" mk_add_options CXX="ccache /Developer/usr/bin/g++-4.2" export CXX="ccache /Developer/usr/bin/g++-4.2" mk_add_options CROSS_LIB_PATH="/Developer/usr/lib:/Developer/SDKs/MacOSX10.5.sdk/usr/lib" export CROSS_LIB_PATH="/Developer/usr/lib:/Developer/SDKs/MacOSX10.5.sdk/usr/lib" mk_add_options CROSS_COMPILE="1" export CROSS_COMPILE="1" export LD=/Developer/usr/bin/ld export AR=/Developer/usr/bin/ar mk_add_options PATH="/home/bzbarsky/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/Developer/usr/bin:/Developer/SDKs/MacOSX10.5.sdk/usr/bin" export PATH="/home/bzbarsky/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/Developer/usr/bin:/Developer/SDKs/MacOSX10.5.sdk/usr/bin" ac_add_options --target=i686-apple-darwin9 ac_add_options --enable-application=browser ac_add_options --with-macos-sdk=/Developer/SDKs/MacOSX10.5.sdk ac_add_options --disable-crashreporter ac_add_options --disable-optimize ac_add_options --enable-debug ac_add_options --disable-tests
Various other --enable-whatever are hanging out in there too, but the above are sort of the minimal set one needs at the moment.
January 29, 2010 10:25 PM
Robert O'Callahan -- H.264 Licensing And Free Software
I've read comments on the Web suggesting that since the MPEG-LA's patent licensing documentation only mentions playback products that are "sold", the MPEG-LA doesn't expect software that is given away at zero cost to need a license. Intrepid LWN reader Trelane actually bothered to ask them, and got a response. Here it is.
The most important part of the response:
In response to your specific question, under the Licenses royalties are paid on all MPEG-4 Visual/AVC products of like functionality, and the Licenses do not make any distinction for products offered for free (whether open source or otherwise).
Also
I would also like to mention that while our Licenses are not concluded by End Users, anyone in the product chain has liability if an end product is unlicensed. Therefore, a royalty paid for an end product by the end product supplier would render the product licensed in the hands of the End User, but where a royalty has not been paid, such a product remains unlicensed and any downstream users/distributors would have liability. Therefore, we suggest that all End Users deal with products only from licensed suppliers. In that regard, we maintain lists of Licensees in Good Standing to each of our Licenses at http://www.mpegla.com.
In other words, if you're an end user in a country where software patents (or method patents) are enforceable, and you're using software that encodes or decodes H.264 and the vendor is not on the list of licensees, the MPEG-LA reserves the right to sue you, the end user, as well as the software vendor or distributor.
January 29, 2010 08:00 AM
January 26, 2010
Gervase Markham -- Protecting Germans I: The Problem
Recently the German government, along with the governments of several other countries, recommended (link in German) moving away from IE to "an alternative browser", due to some well-publicized IE security flaws. Blog of Metrics has shown us the effect this recommendation has had on downloads of the German localized version of Firefox - 300,000 extra copies in one weekend.
Unfortunately, we strongly suspect not all Germans will be getting the real deal. Germany is a place where Mozilla has a particular problem with trademark infringement of various types. The most typical of these are:
- infringing domain name registrations (e.g. typo domains such as "morzilla.de", "firfox.de")
- misleading online content (people pretending to be the official Mozilla download site)
- offers of modified versions of Mozillaâs software, e.g. with a changed toolbar, or containing malware
- "subscription traps" - where something which seems free actually ends up costing you
I've written about subscription traps in the past. These are fraudulent websites which lead users to believe that they are obtaining a typical free download of Firefox or Thunderbird. Only the small print mentions that by registering for the download, the user undertakes a contractual obligation to pay up to £95 (€105; US$150) per year.
This sucks.
Over the next few days I'm going to be blogging about what Mozilla is actively doing to use our trademarks to protect Germans, and residents of other countries as well, from being ripped off in this way.
My thanks to Anthonia Zimmermann of Lovells for her help in preparing this series of blog posts.
January 26, 2010 04:28 PM
Boris Zbarsky -- H.264 patents outside the US
I've seen a number of comments recently along the lines of "I'm in Europe, so the H.264 patents don't apply to me; why are you not letting me have a browser that plays my Youtube videos?" So I took a look at the list of patents on H.264. Or rather, the first 6 pages of the 43 page list. Excluding the US, there are relevant patents granted in at least the following countries:
- Europe: Germany, France, UK, Finland, Italy, Sweden, Belgium, Bulgaria, Liechtenstein, Austria, Czech Republic, Denmark, Spain, Hungary, Ireland, The Netherlands, Poland, Romania, Portugal, Slovenia
- Asia: Japan, China, South Korea, Hong Kong, Singapore, Taiwan, India
- Americas: Canada, Mexico
- Australia
I wasn't reading very carefully; I'd be suprised if I didn't miss a few, or if a few more don't come later in the list somewhere.
January 26, 2010 03:47 AM
Mike Pinkerton -- Task Management, or "Why I'm Neurotic And Inches From The Loony Bin"
January 26, 2010 12:23 AM
January 25, 2010
Asa Dotzler -- spring cleaning for firefox
It's not quite Spring yet, but with the release of Firefox 3.6, now would be a great time to give your browser a good cleaning.
First, make sure you're on the latest Firefox. In the Help menu, select Check for updates... and apply any updates offered.
Next, let's look at your extensions. Go to Tools -> Add-ons... and select the "Extensions" tab. Scroll through your list of extensions and if you see one that you're not using regularly, go ahead and click the Disable button. This won't remove the add-on, it'll just stop it loading up when you start Firefox. You can always re-enable it if you decide you need it back. Next, look and see if you have any extensions that you disabled in the past that you probably won't be using again. Let's remove those. Simply click the Uninstall button.
Go ahead and restart Firefox at this point. You might already notice a big speed-up in Firefox's start-up time.
Now that we've taken care of extensions, let's look at your plug-ins. Plug-in vendors are notorious for adding plug-ins to Firefox without asking you so you might have a few there you don't need or want. To check, go to Tools -> Add-ons again. This time, select the "Plug-ins" tab. Scroll through that list. In my experience the only plug-ins that are really useful are Flash and Silverlight. (and the "Mozilla Default Plug-in"). Unless you know you need it, go ahead and disable any other plug-ins in the list. This will probably help Firefox perform better and certainly keep you safer. You can always re-enable the plug-in if you find that you need it later.
Go ahead and restart Firefox so these plug-ins get unloaded from your browser.
One final clean-up I like to do is to "vacuum" my history and bookmarks database. There is an extension called Vacuum Places that makes this really easy. Once you install the extension and restart Firefox, you should see a button in your statusbar that will trigger the clean-up. You should only have to do this once (and you can then disable or uninstall the add-on) because Firefox 3.6 now does its own periodic cleaning. This will just give you a jump so you don't have to wait on Firefox to do it automatically.
With those simple steps, you'll probably have a much faster, more stable, and overall happier Firefox.
January 25, 2010 08:35 PM
Gervase Markham -- FOSDEM 2010
FOSDEM 2010, the premier grass-roots open source conference in Europe, is fast approaching. Every year, I try and persuade the organizing committee to reduce the number of Mozilla talks so that I can take the time to see what other free software projects are up to. Particularly as we had MozCamp EU last year, I hoped that this year it might actually happen. But, as in every previous year, they have managed to find too many interesting people with interesting things to say :-( Ah, well. As well as all of them, I'll be giving a fast Foundation update, and doing a lightning talk on the Bugzilla API.
Speaking of which: if you are coming, and would love (a maximum of) 5 minutes to tell the Mozilla community about your latest project or ask for help with an upcoming idea, then why not propose a lightning talk of your own?
January 25, 2010 04:25 PM
January 24, 2010
Robert O'Callahan -- ActiveX All Over Again
Many responses to my post about video codec issues have expressed the opinion that "Mozilla should do whatever is most useful for users today, regardless of 'politics' or 'ideology'" and also that "if a user already has codecs on their system, it's wrong for Mozilla to refuse to use them". These opinions fail due to various legal and technical issues that I've already discussed, but I also want to point out that taking such positions is nothing new for Mozilla and history has proved us right for doing so, in particular regarding ActiveX and Web standards in general.
Perhaps it's not widely known, but Gecko has had code to support hosting ActiveX controls, dating back as far as 1999. ActiveX controls are very much like system video codecs. ActiveX support would have been very useful to users ever since 1999, and still would be now --- certainly in corporate intranets, and everywhere in China and South Korea. Enabling ActiveX support would probably boost our market share significantly. Most users have useful ActiveX controls on their machines. But for the last ten years, even during Mozilla's most desperate days, we have consistently refused to turn this feature on, because we believe that ActiveX is not good for the Web.
I think history has proved that this decision was completely right. Our market share rose anyway. The ActiveX ecosystem was a big vector for security attacks. Most importantly, if we'd caved, the Web everywhere would look like it does in China and South Korea, but more so --- dependent on ActiveX, and tied to Windows. No resurgent Apple, no Linux netbooks, precious few Linux users, no ChromeOS, no iPhone, no usable browsers on phones at all, and Microsoft's grip on the industry stronger than one dare imagine. We would have sacrificed huge long-term wins for users --- ALL users, not just Firefox users --- for the sake of a temporary filip.
During the ascendancy of IE, we had similar pressures over the issue of whether to follow Web standards or focus on IE compatibility. The "pragmatists" wanted us to focus on IE compatibility for the very sensible reason that authors had to develop for IE anyway, so it would be easier and cheaper for them if we just fell in line. Obviously pursuing IE compatibility would also have been good for our market share --- and our users --- at the time. We chose standards. Again, I think history shows we made the right decision.
I'm not suggesting that the consequences of exposing system codecs to the Web would be identical to exposing ActiveX. That's unlikely, and unknowable. But favouring our principles over short-term gains for users is nothing new for Mozilla, and when we've done it in the past, history shows it was the right thing to do.
January 24, 2010 10:28 PM
January 23, 2010
Mike Shaver -- HTML5 video and codecs
Recently, Vimeo and YouTube announced that they were moving to support the HTML5 video tag, as DailyMotion did last summer. This is an important step in making video a first-class citizen of the modern web, and that is great news. Unlike DailyMotion, however, Vimeo and YouTube chose to rely on the patented H.264 video encoding, rather than an unencumbered encoding like Ogg Theora. This means that the <video> pages on those sites will not work with Firefox.
Vimeo and YouTube seem to believe that reliance on proprietary plugins for video is a problem on the web. Mozilla believes that reliance on patent-encumbered formats is a problem on the web. Who’s right? Both groups are, in this case; that we can attack, from different perspectives, the multifacted problem of freeing video on the web is an example of the distributed innovation that has made the web such a powerful and popular platform.
For Mozilla, H.264 is not currently a suitable technology choice. In many countries, it is a patented technology, meaning that it is illegal to use without paying license fees to the MPEG-LA. Without such a license, it is not legal to use or distribute software that produces or consumes H.264-encoded content. Indeed, even distributing H.264 content over the internet or broadcasting it over the airwaves requires the consent of the MPEG-LA, and the current fee exemption for free-to-the-viewer internet delivery is only in effect until the end of 2010.
These license fees affect not only browser developers and distributors, but also represent a toll booth on anyone who wishes to produce video content. And if H.264 becomes an accepted part of the standardized web, those fees are a barrier to entry for developers of new browsers, those bringing the web to new devices or platforms, and those who would build tools to help content and application development.
Some companies pay annually for H.264 licenses, which they can pass on to users of their software. Google has such a license, but as they have described, it does not extend to people building from their source or otherwise extending their browser. (Apple and Microsoft are licensors to the MPEG-LA’s AVC/H.264 patent pool, so their terms may differ substantially.) Personally, I believe that it is completely their right to make such a decision, even if I would prefer that they made a different decision.
Mozilla has decided differently, in part because there is no apparent means for us to license H.264 under terms that would cover other users of our technology, such as Linux distributors, or people in affiliated projects like Wikimedia or the Participatory Culture Foundation. Even if we were to pay the $5,000,000 annual licensing cost for H.264, and we were to not care about the spectre of license fees for internet distribution of encoded content, or about content and tool creators, downstream projects would be no better off.
We want to make sure that the Web experience is good for all users, present and future. I want to make sure that when a child in India or Brazil or Kenya discovers the internet, there isn’t a big piece of it (video) that they can’t afford to participate in. I want to make sure that there are no toll-booth barriers to entry for someone building a whole new browser, or bringing a browser to a whole new device or OS, or making and using tools for creating standard web content. And I want that not only altruistically, but also because I want the crazy awesome video (animation, peer-to-peer, security, etc.) ideas that will come from having more people, with more perspectives, fully participating in the internet. The web is undeniably better for Mozilla having entered the browser market, and it would have been impossible for us to do so if there had been a multi-million-dollar licensing fee required for handling HTML, CSS, JavaScript or the like.
I very much believe that Google (both the Chrome and YouTube teams), Vimeo and many others share our desire to have a web with full-featured, high-performance, unencumbered, natively-integrated video, and I look very much forward to us all working — together and separately — towards that end.
People have raised questions about using existing support for H.264 (or other formats) that may already be installed on the user’s computer. There are issues there related to principle (fragmentation of format under the guise of standardized HTML), to effectiveness (about 60% of our users are on Windows XP, which provides no H.264 codec), to security (exposure of arbitrary codecs to hostile content), and to user experience (mapping the full and growing capabilities of <video> to the system APIs provided); I’ll post next week about those in more detail, if others don’t beat me to it.
January 23, 2010 05:45 PM
Robert O'Callahan -- LCA
At LCA I found it harder to talk to people than I normally do at conferences, probably because a lot of people already seemed to know each other and I don't already belong to any of those cliques. The same was true at my first computer science conferences, but there I at least had connections to other students and faculty from CMU that I could leverage. At LCA I was often forced to initiate conversations with random people, which is definitely a good thing.
I went to several interesting talks, and some less interesting. Although I enjoyed the joke talks at the time, I think they probably weren't a good use of time. Some of the other talks were just reiterating things I already knew.
One talk that was really useful was the talk about Clutter. I had been wondering if we should be using Clutter somewhere instead of writing our own layers system, but listening to the talk and talking to Emmanuele afterwards, it became clear that would not work. One big design issue is that Clutter animation runs on the main GLib thread, synchronously with application processing (and that will be hard to change), whereas we eventually need to run animation on its own thread. Another big issue is that Clutter depends heavily on OpenGL whereas we need the option to use D3D on Windows.
Jeremy Allison's talk about Microsoft was good. We've feared Microsoft for so long it's become almost unfashionable, but I think Jeremy is right to keep reminding the free software community of the danger there. He talked about Microsoft's attempts to take over the Web, and kindly mentioned Firefox's role in pulling us back from that brink. He made the point (which I think is too often overlooked) that which company one works for is almost always an individual moral choice and we should hold people accountable for it ... we can't let people off the hook by saying "oh, the company I work for is just evil and I can't do anything about it". The focus of his talk was the suggestion that Microsoft is gearing up for an all-out patent war on free software. I don't know if this is true --- honestly, I expected them to do it long ago and I'm not sure what's been holding them back --- but we certainly do need to keep aware of the possibility. Jeremy suggested that Microsoft will promote "RAND" standards --- standards covered by patents whose licenses would require a "Reasonable And Non-Discriminatory" fee, which sound good except that for free software, any non-zero fee is a show-stopper. In fact, as I discussed later in my talk, RAND-encumbered standards won't fly in the traditional Web standards world --- e.g. CSS and HTML5. We have a very good situation there, where everyone understands that any suggestion that can't be implemented in Gecko (MPL/LGPL/GPL) or Webkit (LGPL) is simply a non-starter. However, we do face a very serious situation in video, where the licensing isn't even RAND, and possibly in other technologies such as touch interfaces. It was good to be able to use some of these issues that Jeremy raised as launching points for my talk.
I went to Jan Schmidt's talk about GStreamer. Although the talk wasn't super-interesting, during LCA I did have some interesting discussions with people about whether we should be using GStreamer as the basis of our implementation of HTML5 <video> and <audio>, like Opera is. For now I tend to think it still makes sense to do our own thing, since there's a bunch of stuff GStreamer has that we don't want, like filter graphs and a GLib dependency. But going forward, as we want to add functionality, I think it's definitely something we shouldn't rule out.
Another interesting talk was Andrew Tridgell on patent defence for free software. Most of it was basic stuff about patents that I already knew, but he made one very important suggestion, which was that we need to find a way for free software organizations to collaborate on patent defence by sharing legal information. Right now this is basically impossible; for reasons I don't understand, the legal opinions we get, we are required to keep secret. This makes us vulnerable to a divide-and-conquer strategy because we can't band together to work together effectively on strategy and to share legal advice to keep costs down. Some sort of legal hacking may be required to solve this problem.
Of course, one of the main purposes of conferences is to meet people, and I did. In particular it was great to meet more of the Xiph/Annodex people behind Ogg. It was also great to meet Carl Worth and talk to him about cairo and the layers work that we're doing for hardware acceleration.
The social events were so-so for me, largely because of the issues I mentioned at the beginning, except for the final Penguin Dinner, which was great. The show was amazingly good ... to be honest, I often suspect "culture on display"-style shows of being insulting to everyone involved, but Friday night's was just great. One nice feature of the dinner was that the food was very good and they put out food for every seat at the table, including the seat next to me where nobody was sitting, so I had a double helping of everything I liked. The only really annoying thing about the Penguin Dinner was the relentless fundraising. OK, Life Flight is a good cause, but we can give to good causes without being constantly harangued ... can't we?
January 23, 2010 10:08 AM
Video, Freedom And Mozilla
Note: below is nothing but my own opinion as a developer of video-related Mozilla code!
My LCA talk on Friday was about why open video is critically important to free software, and what Mozilla is doing about (plus a discussion of the relationship between Web standards and free software in general). Little did I know that Youtube and Vimeo would pick the day before my talk to cast a glaring spotlight on the issue!
Youtube and Vimeo have started offering video playback using the HTML5 <video> element. That is good news for free software, since it means you don't need a closed-source Flash player to play the video [1]. However, they only offer video in H.264 format, and that is not good news for free software. A lot of people have noticed that Firefox doesn't support H.264, and apparently many people don't understand why, or know what the problems are with H.264. This is a good time to restate the facts and re-explain why Firefox does not support H.264. I'll be mostly recapitulating the relevant chunks of my talk. (Hopefully a full recording of my talk will become available from the LCA site next week.)
The basic problem is simple: H.264 is encumbered by patents whose licensing is actively pursued by the MPEG-LA. If you distribute H.264 codecs in a jurisdiction where software patents are enforceable, and you haven't paid the MPEG-LA for a patent license, you are at risk of being sued.
So why doesn't Mozilla just license H.264 (like everybody else)? One big reason is that that would violate principles of free software that we strongly believe in. In particular, we believe that downstream recipients of our code should be able to modify and redistribute it without losing any functionality. This is freedom that copyleft licenses such as the GPL and LGPL (which we use for our code) are intended to ensure. It is possible to obtain patent licenses in a way which works around the letter of the GPLv2 and LGPLv2, but honoring the letter while violating the spirit is not a game we are interested in playing.
But aren't there (L)GPL implementations of H.264? Yes, but they're not as free as they appear. Their freedom has been silently stolen by patents (in jurisdictions where those patents exist and are enforceable). The software license permits you to redistribute and use the code, but the MPEG-LA can still stop you. [2]
But the MPEG-LA won't bother suing me or my project, we're not worth bothering with. Perhaps true, but I hope "remain irrelevant" is not the favoured strategy for most free software projects. It's certainly not an option for Mozilla. If we hadn't distributed Firefox to tens of millions of users --- legally --- it probably wouldn't be possible to browse the Web today using anything but IE on Windows. Plus, relying on selective enforcement is rarely a good idea.
Mozilla should just ship without licensing as a civil disobedience measure. That might be fun, but I expect an injunction would quickly force us to disable H.264 and send a hefty damages payout to the MPEG-LA. That's not a win.
Mozilla should pick up and use H.264 codecs that are already installed on the user's system. I've previously written about a variety of reasons this would be a bad idea, especially on Windows. Really there are two main issues:
- Most users with Windows Vista and earlier do not have an H.264 codec installed. So for the majority of our users, this doesn't solve any problem.
- It pushes the software freedom issues from the browser (where we have leverage to possibly change the codec situation) to the platform (where there is no such leverage). You still can't have a completely free software Web client stack.
But I could just download gst-plugins-ugly and I'd be OK. That's a selfish attitude. Everyone should be able to browse the Web with a free software stack without having to jump through arcane hoops to download and install software (whose use is legally questionable).
The H.264 patents will expire soon, and then we'll be OK. Many H.264 patents don't expire until 2017 or later. Anyway, H.264 isn't the last word in video compression. There will be an H.265 and the same set of problems will persist.
Users just want video to work. You Mozilla people are such idealists! Yes, that is the reason for Mozilla to exist. Anyway, in the short term, our users probably won't be affected much since Flash fallback will still work. In the long term, I think freedom will ultimately benefit users (not just Firefox users, but all users).
Apart from the issues with H.264 support in clients, there are also huge issues around H.264 for Web authors and content providers. Currently providing H.264 content on the Internet is zero-cost, but after 2010 that will almost certainly change. A couple of good articles are here and here. We won't know much about the terms until the end of this month. The key issue is not exactly how much it will cost, but that if you want to publish H.264 you will probably have to hire lawyers and negotiate a license with the MPEG-LA. If you just want to put a few videos on your Web site, or add a help video to your Web application, or put a video cut-scene in your Web game, that is probably not something you want to do. Web video is not just about Youtube; mandatory licensing would cripple the use of video on the Web. (Just imagine if we had such a regime for still images...) Even if there were no patent issues on the client side, this would still be a good reason for Mozilla to push for truly free codecs.
The honest truth is that none of us know how this is going play out. The proponents of mandatory licensing are strong, and most people don't care about software freedom. We're doing our best to make Ogg Theora rock, and I don't know what else we can do directly right now except spread the word and help people understand what's at stake here ... hence my LCA talk, and this blog post.
[1] Yes, I know gnash and/or swfdec might play these videos, but in general they are not able to keep up with the latest Flash APIs offered by Adobe and used by major sites. Anyway, it's good to not have to play that game.
[2] RANT: in many cases, free implementations of heavily patent-encumbered technology are harmful to the free software ecosystem, for two reasons: people are confused into thinking they have rights that they actually don't, and thus these implementations can discourage the adoption of alternatives that are free for everyone.
January 23, 2010 07:51 AM
The Road To LCA: Ruapehu
Several Mozilla-related people went to linux.conf.au in Wellington last week. We decided to combine this with our annual outdoorsy "team building for accounting purposes" Auckland Mozilla office event, by driving down from Auckland and spending a couple of days in Tongariro National Park on the way.
So, Sunday night found me, Karl Tomlinson, Michael Ventnor, Josh Matthews, Taras Glek, Matthew Gregan, Chris Pearce, Tim Terriberry and a few others in National Park. The drive down was great ... our visitors finally saw the sheep they'd been waiting for. We had a great dinner at Schnapps Bar, the only hitch being that we exhausted their venison pie special. Our evening was occupied with a six-player game of Settlers. I won.
On Monday the weather report predicted afternoon thunderstorms so our plans to ascend to the summit of Mt Ruapehu were aborted. Instead we took the Whakapapaiti Valley Track from the Bruce Road down to Whakapapa Village, many of us taking a detour to the Silica Rapids near the end. Even though it rained a fair bit, I still really enjoyed this walk. It descends from alpine terrain into forest, crosses some nice mountain streams, and the Silica Rapids look awesome. Photos below.
Monday night's dinner was at the Station Cafe ... pretty good, although I think Schnapp's Bar is better value for money. Monday night's game was Scrabble. Josh won, although I claim I was crippled by getting 4 "I"s on my last tile draw.
On Tuesday the weather was again very cloudy up the mountain and the chairlifts were reported to be closed. Just for the heck of it we drove up to the Whakapapa ski area anyway, and thanks to some fast talking by Karl we managed to hire a guide and get the chairlifts to run us up the mountain to the start of the walk to the Ruapehu summit. Due to people leaving on Monday night and Michael not feeling up to wearing his boots for another five hours, six of us went up: me, Matthew and Casey, Karl, Josh and Taras. It was a fantastic trip. The cloud came and went, but we had great views of parts of the mountain, especially the crater lake. Just being up there eating lunch looking down into the swirling waters of the lake was amazing for a volcanophile like me. (The base of the lake is a molten sulphur plug over the vent; the surface lake water is currently heated to 16-17C and is highly acidic.) The walk up was steep and rockhoppy, but fun, and going down was even more fun, due to a few short cuts that involved butt-sliding down the snow. Highly, highly recommended. I can't wait to do it again.
Our guide Callum was great, and had a lot of stories to tell. He was involved in the rescue operation in 2007 when the volcano belched a few rocks into the Dome Shelter hut and smashed the leg of a climber sleeping there. The DOC ranger we met at the summit claimed that there are still pieces of sleeping bag at the bottom of a hole in the hut floor...
As soon as we got back to the car, Michael, Josh, Taras, Karl and I drove straight to Wellington. We didn't get there in time for the Speaker's Dinner, but I had already expected that ... I heard the dinner was great, but I don't regret choosing Ruapehu instead! I'll blog more about LCA soon.










January 23, 2010 05:52 AM
January 22, 2010
Eric Meyer -- MIX Judging
I was recently honored to be asked to be a judge for the MIX 10k Smart Coding Challenge, running in conjunction with Microsoft’s MIX conference. The idea is to create a really great web application that totals no more than 10KB in its unzipped state.
Why did I agree to participate? As much as I’d like to say “fat sacks of cash“, that wasn’t it at all. (Mostly due to the distinct lack of cash, sacked or otherwise. Sad face.) The contest’s entry requirements actually say it for me. In excerpted form:
- The entry MUST use one or more of the following technologies: Silverlight, Gestalt or HTML5…
- The entry MUST function in 3 or more of the following browsers: Internet Explorer, Firefox, Safari, Opera, or Chrome…
- The entry MAY use any of the following additional technology components…
- CSS
- JavaScript
- XAML/XML
- Ruby
- Python
- Text, Zip and Image files (e.g. png, jpg or gif)
Dig that: not only is the contest open to HTML 5 submissions, but it has to be cross-browser compatible. Okay, technically it only has to be three-out-of-five compatible, but still, that’s a great contest requirement. Also note that while IE is one of the five, it is not a required one of the five.
I imagine there will be a fair number of Silverlight and Gestalt entries, and I might look at them, but I’m really there—was really asked—because of the HTML 5 entries. By which I mean the open web entries, since any HTML 5 entry is also going to use CSS, JavaScript, and so on.
The downside here is that the contest ends in just one week, at 3pm U.S. Pacific time on 29 January. I know that time is tight, but if you’ve got a cool HTML 5-based application running around in your head, this just might be the time to let it out.
January 22, 2010 08:50 PM
Asa Dotzler -- firefox 3.6 awaits you
Help -> Check for updates...
Or, if you're on some other browser, just head over to getfirefox.com
January 22, 2010 12:41 AM
January 20, 2010
Postbox Team -- Indie+Relief Effort to Aid Haiti
Today, nearly 150 software developers are participating in Indie+Relief, an effort to help the people of Haiti who are in desperate need of aid. Proceeds from today’s sales will be donated to a variety of charities and support organizations, such as the American Red Cross, Doctors Without Borders, UNICEF and more.
Visit Indie+Relief today to purchase some terrific software and to help those in need.
January 20, 2010 08:45 PM
Ian Hickson -- T-Mobile makes no sense
For reasons that aren't material to this rant, I'm trying to get a T-Mobile SIM card set up with T-Mobile's ridiculously named "Even More Plus 500 Talk + Text + Web" plan. At this point I have a SIM card, and after much pain and suffering in phone conversations and online chat support, I have an account that is fully paid up.
Let's log in to the "My T-Mobile" site's "Billing & Payments" page to see how much I currenly owe:

Ok it says I owe $62.42 by 1/19/2010 (today) for service from 2/9/2010 - 3/8/2010 (next month). But it also says that my monthly charges for my previous bill are $127.42 due on 3/9/2010 (two months from now).
None of those numbers seem to add up to $59.99, which is what the "Even More Plus 500 Talk + Text + Web" plan is advertised as costing, but I suppose $62.42 could equal $59.99 if one were to imagine that T-Mobile felt they could put a few "fees" on there that for whatever reason could be assumed not to count. But what's with the $127.42? And what's with the due dates? Let's click "See details" to see details:

Wait, bill summary? I wanted details! And hold on, this page says my account balance is $0.00, and that I paid $62.42 on 1/20/10 (tomorrow). It also gives my monthly charges twice... once as $0.00, and once as $127.42! I guess if you average them it adds up to $63.71, which is similar to $62.42...
Oh look, the details are below the fold.

This says the next service period is $63.71, that they've received $62.42, that the previous balance was $63.71, and that they "adjusted" my account by $65.00 (the result of my calling billing support because they hadn't applied the $63.71 they actually charged me to my account), and apparently that all adds up to me owing $0.00... by 02/05/2010.
Clearly maths is not their strong point. Also not their strong point: being consistent about how they mark up dates. Or how they decide what dates are important.
Oh and... also not their strong point: matching reality. So far they've only actually charged me $63.71, once, though I have in fact tried to pay more than once (earlier today I also tried to pay the $62.42 that some of these pages claim I owe, though my bank has yet to see that charge).
Still, the $127.42 is a bit confusing. Let's see what the "View Bill" button under that number brings up.

Whaaaa!? FPEvenMorePlus 500TTW $119.98? What on earth is that? "Amount due 2/05/10: $127.42"? That completely contradicts the previous page! How much do I owe on that day, $0, or $127.42??
They say the way to work out how much you owe is to dial #225# from the phone, so let's try that... Ok, that says $0. I guess the phone would know best.
I give up trying to work out how much I owe; let's instead try to get the "Unlimited Data" working on this phone. I've heard rumours that one has to buy a $0 optional extra "Unlimited Web" service since otherwise the "Even More Plus Unlimited Web" is actually limited. Add additional services on this line... Internet & E-mail... Aha, here we go. "T-Mobile Smartphone Unlimited Web with FlexPay". That sounds promising. And it's free! Just like the rumours. Let's "buy" that and quickly review the charges:

Added plan and services... $0, yup. Unchanged plan and services... $59.99. Yup. So, the grand total is... $89.99. Wait, wait, hold on, let me recheck the maths here... $0... plus $59.99... carry the one... What??
Ok, ok, clearly it got confused. Let's close the browser and start over. Add additional services on this line... Internet & E-mail... "T-Mobile Smartphone Unlimited Web with FlexPay". And it says FREE. So we click it again, and let's see how much $59.99 + FREE equals this time:

TEN THOUSAND AND EIGHTY EIGHT DOLLARS AND NINETY NINE CENTS?!
I give up. I'll try calling them tomorrow.
January 20, 2010 08:02 AM
Boris Zbarsky -- From the mouths of babes
Today Arlan happened to look at my screen when part of the desktop was actually visible. That led to approximately the following conversation:
- Arlan:
- Папа, там лиса!
- Me:
- Где? Да, там лиса.
- Arlan:
- Еще лиса! Две лисы.
- Me:
- Две?
- Arlan:
- Один, два, три.... Три лисы!
Little did he know that the fourth one (3.5 to be exact) was hiding under the iTunes window....
January 20, 2010 05:19 AM
January 19, 2010
Alex Vincent -- XPCOM and Logging, part 2
First off, thanks to everyone who took the time to reply to my original blog post. Several people pointed me to moz4js, and I'm quite pleasantly surprised to find out the Mozilla Thunderbird team uses it quite a bit. I hadn't heard of it, but when I did through the comments, I found it had the same problem NSPR logging has: single language support.
Then I realized: if we put moz4js in a JavaScript module, then a component can reference that module and expose the same interfaces to XPCOM. Chrome JS and JS components can load the module, and all other languages can use the XPCOM component. Maybe not simple, but certainly workable.
No, I'm not signing up to own the toolkit moz4js bug. I'm not wholly convinced, personally, it's the right way to do logging. What I do think, though, is that we need a general discussion on logging API's and consumers.
In the bug, Flock mentioned their own API - which I also didn't know about. I mentioned Skyfire has an internal logging system. It makes me wonder how many other companies and projects (Songbird, Komodo, Disruptive Innovations, etc.) have implemented their own logging API's. This is one subject that reasonably affects the whole Mozilla developer community.
If MozPad were still active, this is one subject I'd put right to them and develop a standard logging API for. Since we don't have MozPad anymore, I would like to put it to the community at large:
- What do you require logging for?
- What logging capabilities do you need?
- What logging infrastructure (API, code) have you already implemented or used?
- What tests exist and pass for the logging infrastructure you use?
- How extensively do you use your logging infrastructure?
- How easy is it to work with your logging infrastructure?
- If you use someone else's logging infrastructure, what's missing?
- Are you willing to work on getting your logging infrastructure into the Mozilla code base? (Writing code, tests, specifications)
- Are you willing to work on developing a common Mozilla standard logging API and infrastructure, and getting that into the Mozilla code base? (Writing code, tests, specifications)
- Are you willing to actively "own" and drive an effort towards adding a Mozilla standard logging API to the Mozilla code base?
Please reply in the comments. The quality and quantity will tell me how seriously I should be taking this.
(P.S. I found a bug in my logging interfaces already. Not the JavaScript code, but the IDL. Bonus points to the first person to spot it besides me - it's a little undocumented gotcha.)
January 19, 2010 07:36 AM
January 18, 2010
Asa Dotzler -- the french agree with the germans
Just a few days after the German government warned users off of Internet Explorer, the French government has done the same.
Yes, all browsers have flaws, some of them security flaws. But IE has a known flaw that has been weaponized. There have been and are ongoing attacks in the wild. It is not safe.
Even if only temporarily, please move to another browser.
update: more on this from BBC
January 18, 2010 04:26 PM
Alex Vincent -- XPCOM and Logging
What we have now
Over the last several years, I've learned to appreciate good message logging. Mozilla's NSPR Logging API is a very good example, and works well, for compiled code.
JavaScript components? They can't use NSPR. Neither can chrome, PyXPCOM, or any other privileged code. The best alternative I've got is a dump() statement. dump() writes to the standard error console, commonly known as "stderr". In debug builds, though, stderr gets populated with a bunch of other stuff that I really don't care about at a particular moment. (I'm not entirely clear of blame, either - leftover dump statements from my code pollute that console as much as anything else.)
At Skyfire, I've had to implement a couple logging systems as well, and the generated logs have helped us find and isolate bugs relatively quickly. What you log can help you identify bugs in most cases. They're not a panacea, though: doing it wrong can really cost you.
Over the past few days, I hit a really weird bug in Verbosio's templates system (which I've been quietly working on for about five months now, in a complete rewrite with tests). One test fails at a critical point, but it's pretty far removed from the starting point. Amid all the other dump() statements and debug output, I can't quite figure out where the code started going wrong. To pin it down, I think the best approach is to insert some logging.
Oh, and did I mention that Mochitest includes logging for pass, fail, and todo conditions... but conveniently left out an informational-only logging feature? (Having that helps when you're trying to trace where a log statement came from - and the conditions at the time.)
The five paragraphs above combine to form one idea: Maybe I should implement my own logging service with tests.
Thoughts in the design
- First of all, keep it really simple. Most logging implementations I've seen use a module name, a severity level, and a summary string. I figure that plus an optional details string should cover most messages.
- For where those messages go, files are good. Streams are better. With a stream, you can feed it into a file, or read it into dump(), a XUL page's multiline textbox, or anywhere else.
- Someone reading the logs probably wants a way of separating messages - so if I give them a "multipart" stream, where each part is a separate stream... it's easy to tell each message apart.
- Make sure the user requesting the logging stream can specify which messages they want to receive - both by module name and severity level, and whether or not they want the details with each message.
- Make sure to cache all the messages, so that a logging stream created later still gets all the messages that came before it.
- Finally, since each message ultimately ends up as a simple string - use a simple string stream interface to construct each message as needed.
Did I overdo the design? Probably. On the other hand, it really maximizes flexibility. I don't know how much memory multipart input streams consume, though, so I don't recommend using this in a production environment. If you use this logging service for debugging, make sure you remove all references to it before checking in!
Did it solve my problem?
No.
Oh, the logging itself worked beautifully. But to get some of the data I wanted to log, I had to call other functions. Some of those functions triggered JavaScript garbage collection, which tried to run some cleanup code, which tried to execute JavaScript code I wrote, which resulted in a 100% reproducible crash. I don't yet know what I'm going to do about that. I'll probably first try to reduce the testcase and post it.
Also, logging itself will never solve the problem you've got. All it will do is provide data, and hints about where you are in the code execution. You still have to tell it what and when to log, and you have to read those logs to find the bug.
The good news is, though, that a logging infrastructure like this only took one afternoon to write. Sometimes, that's all I need: a quick, simple component to organize the dirty work into one place.
(The code I wrote I am making available under MPL 1.1 / LGPL 3 / GPL 3 tri-license. If you want it under another license, feel free to e-mail me, and I'll almost certainly say yes. I don't currently plan on submitting this in a bug for landing in mozilla-central, though.)
January 18, 2010 06:01 AM
January 15, 2010
Gervase Markham -- Packages and Trademarks - An Observation
Better to have tried and failed than not to have tried at all, right?
Generations of fathers would say to their kids "of course". But it seems not in every case.
A while back, Mozilla and Debian had several goes at coming to an agreement for the use of the Firefox name and logo. In then end, we collectively failed[0], which is something I consider a great shame. Sadly, this failure seems to have left a significant number of free software people with a residual unhappiness and antipathy towards Mozilla. Which is, in some ways, an even bigger failure.
On the other hand, no-one seems to have even suggested that there's a possibility that the new kid on the block will allow their name to be used when Debian packages their software. So Debian are happily and peacefully packaging it under the alternative name. I haven't noticed anyone being upset about this.
Why is that?
[0] And I'm not interested in reopening the debate about who was right. That's not the point of this post.
January 15, 2010 06:10 PM
A New Use For JavaScript URLs...
I was just invited by a 'friend' to join a Facebook "free laptop" group whose real aim is to get you to take surveys.
Facebook have obviously decided not to have a "Select All" button on your list of friends when you share stuff, to try and reduce spamming. Here's this group's way around that, from their instructions:
3) After that, Click the 'Invite People to Join' link on the left hand side of the page. (Must do it, it's part of how we can give these buttons away for free!)4) Erase everything in your address bar... (The address bar is where http://www.facebook.com is typed, where you type in a website to go to.)
Then copy and paste the following code in there and hit ENTER.
javascript:elms=document.getElementById('friends').getElementsByTagName('li');for(var fid in elms){if(typeof elms[fid] === 'object'){fs.click(elms[fid]);}}
5) Once you've done that, all of your friends in the box should turn BLUE, Click 'Send Invitations'
So, how about a JS URL which creates form elements mimicing a Facebook login form, waits for them to autofill, and then posts the contents off to a server? (Or did we fix that?) Or one which inserts a <script> tag with a src at an attack site? If people get used to blindly following instructions like this, no good will come of it...
January 15, 2010 10:32 AM
January 14, 2010
Robert O'Callahan -- Volcanoes And Penguins
Next week is linux.conf.au in Wellington. I'm going to be there from Wednesday to Friday, and I'll be giving a talk about why open video is important for free software. Actually I think I'll make it more interesting by broadening it with some discussion of the relationship between Web standards and free software in general.
On the way down to Wellington, a group of Mozilla-related people are going to be stopping at National Park Sunday night and Monday night so we can attempt the Mt Ruapehu summit walk. Should be fun, if the weather cooperates.
Anyway, this means I'll be completely offline Monday/Tuesday and mostly offline Wednesday to Friday next week.
January 14, 2010 11:27 PM
More On Patch Division
Kyle Huey asked me to comment on my approach to breaking up work into a large number of small patches.
First a big disclaimer: I've only been doing this for a short time, but I know other communities have been doing it for a long time. I bet Linux kernel developers know far more about this than I do. Probably many Mozilla developers do too.
As I always have, I try to plan out the work into incremental steps in my head before I actually start writing code. This plan always needs modification after contact with the enemy but it generally works. Each step is of course a new mq patch.
Sometimes I find that additional steps are needed. If I need to add new steps after the point I'm currently up to, there's nothing to do. If I need to add new steps before the point I'm currently up to, it's easy to pop off mq patches and do the work in the right place.
When I think I've completed a step, I often (but not always) do a build to make sure it compiles. However, I rarely test my changes until I've finished the work. Perhaps this makes me a bad person, but I find that test effort before that point is often wasted since I frequently go back and change previous patches as I learn things from writing later patches.
When I'm testing and fixing bugs, I accumulate many fixes as a diff in my working copy. Periodically (once a day?) I take the changes in that diff and distribute them to the patches that each change logically belongs to. I.e., if I fix a bug in new code that was added by a patch, I apply the fix to that patch.
Once the code works, I then reread the patches (a sort of self-review). One thing I look for is large patches that could be broken up into smaller ones consisting of logically independent changes. (I especially focus on the largest patches, to try to minimize the maximum patch size.) Wherever breakup is easy to do (e.g. because the smaller patches don't overlap), I'll do it. If it's hard because patches overlap, I generally won't do it unless the gain in clarity seems large. When patches can be broken up into really tiny fragments, I'm not sure what the correct minimum patch size is ... a patch consisting solely of many instances of the same mechanical change probably isn't worth breaking up, unless different reviewers are going to review each piece.
It's important for bisection searches that at each stage in the patch queue, the project at least builds. To verify this for large patch queues I just run a script overnight that does a loop of hg qpush ; make.
When the code is ready to submit, I upload all the patches to the bug, numbering them with "Part NNN" to make it easier to refer to a particular patch. For large projects it's also helpful to publish the patches as a Mercurial repository using qcommit.
If reviews take a while, as they often do for large projects, every so often (once a week to once a month) I'll refresh the patches to trunk (see below) and push the results to tryserver to make sure nothing's broken.
One tip: at least with Mercurial 1.1, DO NOT refresh to trunk using rebasing (e.g. hg pull --rebase) while you have mq patches applied. Instead, hg qpop -a, hg pull -u, and then hg qpush -a. For me, rebasing screws up my patches in various ways. Also I find that fixing conflicts during rebasing is far more tedious than fixing conflicts during qpush, partly because I have no idea what rebasing is actually doing, but mostly because as of 1.1 rebasing with mq seems to force me to fix the same conflicts over and over again.
January 14, 2010 08:51 PM
Asa Dotzler -- html5 and open video for youtube
HTML5 <video> with Ogg Theora codec support seems to be pretty popular among those voting on YouTube feature ideas.
These kinds of voting systems are of course heavily influenced by activist types, but it's still nice to see how dominant this feature request is given all of the other solid ideas in the list.
January 14, 2010 07:18 AM
January 13, 2010
Gervase Markham -- Find A Firefox
The prolific Simon Willison and a happy band of hackers squirrelled themselves away in a fort in October 2008 and built WildlifeNearYou.com. (MySociety site-naming style, perhaps because Matthew Somerville was one of the hackers involved.)
This means I can now point you at a list of places in the world you can go to see a Firefox (a.k.a. the red panda)!
If you know of Red Panda locations not on that list, you need to upload a photo of it to Flickr and follow the instructions in the FAQ. I'm off to add the one in Symbio Wildlife Gardens, Sydney, Australia.
January 13, 2010 12:46 PM
January 12, 2010
Robert O'Callahan -- CSS Absolute Length Units
While reexamining the way we handle screen resolutions in Gecko, we had to reconsider how to handle CSS "absolute length units" --- especially pt, but also in, cm, mm and pc. The spec says that these units should be displayed at their physical sizes, so CSS "1in" is rendered as one inch, etc. Unfortunately this breaks Web content, because most Web pages were designed on desktop screens. A one-inch margin may be fine on paper or on a desktop screen but doesn't make much sense on a two-inch-high phone screen. For related reasons, IE and Webkit have already redefined these units to be fixed numbers of CSS pixels, i.e. 1in = 96px, etc.
After I raised this issue on www-style, it became clear that this is not just an issue of compatibility with existing content. In fact, absolute length units as defined in CSS are only rarely useful. When do you want to force a length to be one inch, no matter what kind of screen is being used or how far from the user's eyes it would be? The only use cases I know of would be touch interfaces and "life size" diagrams. Indeed, the CSS spec says
Absolute length units are only useful when the physical properties of the output medium are known.On the Web, physical properties of the output medium are almost never known, so the spec suggests that currently absolute length units are useless on the Web.
It seems to me that it will be far more useful, as well as compatible with existing Web content, to specify that absolute length units should take those physical lengths when printed to "normal paper" --- paper (or other media) that you hold when reading, like almost every printed page. Then the browser decides how to render the content on a screen to best express the author's intent. Common sense, as well as Web compatibility, will dictate that the ratios between "px", "pt", "in", "cm", "mm" and "pc" will be fixed based on the assumption that 1in = 96px.
We may find it useful to define a new unit or units, say "truemm", that is a true physical millimeter, for the rare use cases of touch interfaces and "life size" diagrams.
Redefining "in" to not always mean physical inches is quite controversial in www-style. I hope sanity prevails.
January 12, 2010 09:37 AM
Unifying Abstractions, Dividing Patches
Speaking of architectural misfeatures, one that's been glaring for quite some time now is the Gecko "view system" --- nsIView, nsIViewManager and friends. Originally views were intended to be used as "heavyweight" rendering objects for elements that used "advanced" features such as translucency. Over time it became clear that it was simpler to just support those features directly in the "frame tree" of rendering objects that most elements have, and now we see that everything in the view system would be simpler and more performant if it was handled by the frame system.
However, there are large chunks of functionality still handled in the view system. One big one is scrolling. Scrollable elements have an nsHTMLScrollFrame (or nsXULScrollFrame) which handles layout, scrollbars, and related stuff, except for the actual scrolling --- the visual movement. That is handled by nsScrollPortView. Well, until today, when I landed a series of patches to move all scrolling functionality in the view system out to the frame system. The hard part of these patches is that there were two interfaces exposed to let other parts of Gecko manipulate scrolling --- nsIScrollableFrame and nsIScrollableView. A lot of code was using the latter view-system interface; APIs present in nsIScrollableView but not nsIScrollableFrame had to be added to nsIScrollableFrame, and then all the callers of nsIScrollableView APIs had to be modified to use the frame API instead. This was a lot of work --- not really very hard, but difficult to get right without accidentally causing regressions.
Anyway, it's done. A lot of code that used to have to mess around with views no longer has to know about them. Scrolling code is simplified since it's all in one place. Page load is little more efficient because we don't have to construct scrollable view objects. Better still, this just happened to fix the long hang at the end of Firefox loading the HTML5 spec, since that was all about updating a big pile of scrollable view objects! (There are still some issues with slow script execution ... they're being worked on.)
When I did this work I decided to do a little experiment in development style by breaking up the work into a very large number of small patches, managed with Mercurial Queues. The final commit has 37 separate changesets. I thought this might be overkill since our culture, and Bugzilla, isn't really designed to handle that many patches for a single bug. While Bugzilla (especially the way we use it for code review) could certainly be much better at handling a large set of patches, overall I declare the experiment a resounding success. I think this approach made it much easier for me to keep track of things, made code review much easier and less daunting, and made it much easier for me to keep the patches up to date over several months as the tree changed underneath it. I'll do it again. In fact, I'm already doing it again with the layers work --- more about that later :-).
January 12, 2010 09:15 AM
There Can Be Only One
For quite a while now, Firefox 3.6 has been done as far as most Gecko work is concerned. Meanwhile all kinds of exciting things are happening on trunk, but we haven't been blogging enough about them. I'll try to do that a bit more starting today, and I encourage my colleagues to do the same!
Since it was first designed, Gecko has supported multiple presentations per document (in code terms, multiple nsIPresShells per nsIDocument). The idea is that you would be able to have multiple views of the same document, each with different style sheets and layouts, and display and even edit the views simultaneously and have them all stay in sync. It sounds cool, and the idea also appears in the DOM specs (see document.defaultView), but the reality is that there is no need for this feature in Web browsers, as Dave Hyatt pointed out a long time ago. (Different views into the same presentation --- e.g., live thumbnails --- are useful, but that's a different and much simpler feature.) Supporting multiple presentations just adds unnecessary complexity.
The only use of multiple presentations in Gecko was for printing and print preview. When printing or previewing a document, we'd create a temporary extra presentation. But for printing you don't need a new presentation for the same document, it's fine to make a copy of the document and create a presentation for the copy. In fact this is a better approach, because you can print or display a preview while continuing to change the original document. You don't want to be displaying a print preview while a script keeps making changes to the document; apart from not being what the user wants, we never supported such dynamic changes properly, so we had to carefully freeze scripting while a print preview was displayed. This was complex and fragile.
Thanks to Olli Pettay this is all history now. He went ahead and implemented the document copying approach for printing and print preview. This approach let us fix a lot of bugs more easily. For example, we can take a snapshot of the current frame of each animated image --- and video, and plugins --- when copying the document, so the copy is truly static (although you can still change the layout, e.g. by changing the page margins). We don't have to worry about temporarily freezing scripts --- we just disable scripting altogether in the copied document.
Better still, this let Olli go ahead and remove the support for multiple presentations. This lets us do various simplifications and optimizations. In particular, before, each presentation had a hash table mapping elements to their "primary frames" used for rendering that element in that presentation. Now a DOM element can have a direct pointer to its primary frame. This is more efficient and also lets us use much simpler code since we no longer need to maintain primary frame maps.
In general there are lots of places in our code where we used to have to think "am I using the right presentation for this document"? We no longer have to think about that.
One architectural misfeature eliminated. Stay tuned for more :-).
January 12, 2010 08:04 AM
January 11, 2010
Boris Zbarsky -- More on distcc
A few days ago I mentioned that I've been experimenting with distcc. I finally took the time to do some timings. All times below, unless stated otherwise, are for a completely clean debug non-libxul build on my Mac. In particular, none of the files are in ccache, but ccache _is_ being used (so the compile time includes the cost of caching the new object files). distcc is being invoked via CCACHE_PREFIX. The distcc builds were done with -j10 (I also tried higher -j numbers and saw things between no effect and a slowdown as I got up close to -j20) and allowing 10 distcc connections to the server. The non-distcc builds were done with -j3, which is my normal build setup (since the mac in question only has 2 cores). distcc over ssh did not use ssh connection sharing, and was doing X forwarding on each ssh connection; I should retest with those two things changed at some point, if I decide I don't like doing things over tcp. The network is a wireless 802.11g on the Mac laptop end and gigabit ethernet to the wireless router on the Linux desktop. I found that ,lzo for the distcc builds didn't materially affect things (may have made the build 1-2% faster when used).
| Build type | Time |
|---|---|
| local build on the Mac | 38 minutes |
| distcc over ssh | 50 minutes |
| distcc over tcp | 32 minutes |
This was obviously distressing. I didn't test distcc over ssh after this, but particularly distressing was the fact that the compilation seemed largely diskbound and therefore distcc didn't help much. This got me started looking into hard drive performance as a function of utilization, since I could swear that the Mac's hard drive has been getting slower as it fills up. Turns out, this is actually the case. So I moved my rarely-used Linux VM to an external hard drive, which took free space on my internal disk from 4GB to 65GB or so. With that change done:
| Build type | Time |
|---|---|
| local build on the Mac | 32 minutes |
| distcc over tcp | 22.5 minutes |
Progress! Next, I tried dropping ccache, which let me measure the impact of pump mode. Since I do generally see a benefit from ccache, and wanted to measure it, I also tried a build without distcc but with a primed ccache (so build a tree, delete the objdir, rebuild that tree). Results:
| Build type | Time |
|---|---|
| no-ccache distcc in pump mode | 16 minutes |
| no-ccache distcc without pump mode | 17 minutes |
| Local build coming entirely from ccache | 15 minutes |
So if I were pretty sure I'd almost always be near this Linux box, just not using ccache and using pump mode might be the way to go. As it is, using distcc + ccache seems like a good approach for now.
Now all I need is a way to figure out a way to get this sort of data without it taking several hours, a way to pick good values for -j, and a faster internal hard drive in this laptop.
January 11, 2010 08:48 PM
SeaMonkey Team -- SeaMonkey 2.0.2 Update
As part of Mozilla's ongoing stability and security update process, SeaMonkey 2.0.2 is now available for Windows, Mac, and Linux as a free download from www.seamonkey-project.org.
We strongly recommend that all SeaMonkey and old suite users upgrade to this latest release. If you already have SeaMonkey 2.0, you will receive an automated update notification within 24 to 48 hours. This update can also be applied manually by selecting "Check for Updates..." from the Help menu.
For a list of changes and more information, please review the SeaMonkey 2.0.2 Release Notes.
Note: All SeaMonkey 1.x and old Mozilla or Netscape suite users are encouraged to upgrade to SeaMonkey 2.0.x by downloading it from www.seamonkey-project.org.
January 11, 2010 07:08 PM
Gervase Markham -- Committers Whose Email Address Has Changed
If you have Mozilla commit access, and your username corresponds to an email address which no longer works, you will have missed important emails about signing the new Mozilla Committer's Agreement. This means
your commit access may go away at any time
(don't make me turn on 'blink') which would be bad. If this is you, contact me, and then follow these instructions, remembering to note on your submitted form the username of your account. And perhaps also file a bug on IT on updating your username.
January 11, 2010 02:36 PM
Asa Dotzler -- firefox add-ons
In case you don't read newsgroups, here's some of Mike Beltzner's latest comment:
The JetPack project is very interesting to Firefox, and we are participating (as should all of you, as interested consumers and stakeholders!) in its development both in terms of building the platform and setting requirements on that platform. It is a stated goal of the Firefox team to do what we can to make it easy to install, uninstall and manage JetPacks alongside other customizations such as Add-Ons, Plugins, Search Providers, Themes, and Personas.We also plan on continuing to dedicate time to working with the JetPack drivers to help them shake out requirements based on existing Add-Ons and what we know the most common compatibility "gotchas" are. It's my sincere hope that mconnor's "call to arms" will be heeded and that everyone contributes to that effort. JetPack is being built specifically for the purpose of making it easier to write customizations for Firefox, so please, tell us what you find difficult and help us understand what your requirements are.
Many people have interpreted mconnor's post to mean that the Firefox product team has already made a strategic decision to remove support for the existing extensibility platform and technology in the near future: this is not the case. What mconnor wrote was that from a strategic perspective, we will be focusing on building an effective and viable alternative platform in conjunction with our extension development community in order to address specific limitations of the current system. I do not see this as a zero sum game, although as a separate but related issue, some specific aspects of the current model may need to be modified for security and performance reasons (I'm referring to binary components, which will likely need to migrate to js-ctypes.)
I hope this clarifies things, somewhat.
And BZ chimes in as well:
I think this hasn't been made clear enough, actually, so here's my attempt to make it clearer.Right now we seem to have on the order of 10000 (a bit over 11k if I add my numbers right, but it's the order of magnitude that counts) Firefox add-ons on AMO.
Some of these are actively maintained; some are not.
The really popular add-ons, in my experience, have a tendency to be actively maintained. Even on trunk, adblock+ usually works just fine, and more importantly says so in its version metadata. Same for Firebug (it's been worse in the past, but is getting better). That sort of thing.
However there is a very long tail of add-ons that are not as actively maintained. This is fine; people are doing many of these in their spare time, and even the time they take to bump the maxVersion every time we do a new release is much appreciated. But the net effect is that:
1) Possible beta testers tend to avoid trunk builds because their
favorite add-on or three doesn't work with them yet, so we lose
useful feedback. The version check override approach helps some,
but only if the version check is really the only thing preventing
the add-on from working.
2) Doing a release involves getting several thousand people to update
things on AMO (at least the maxVersion, but in many cases more than
that).
3) Since even the updates from step 2 are not enough for the whole
long tail, we end up with users who are running outdated (hence
insecure!) versions of the browser because their favorite add-on
hasn't been updated.If we can get to a point where we serve, say, 98% of these add-ons with a stable API that doesn't involve any more changes, and if the most popular of the remaining 2% are updated expeditiously anyway, we suddenly find ourselves with a _much_ shorter tail, and the three issues above become much smaller problems. Note that this doesn't involve all extensions using the stable API, nor does it involve preventing truly innovative extensions from being written, nor anything along those lines. It just involves a preponderance of extensions using the stable API. Note that the stable API can have things added to it as needed; this would probably happen based on feedback from extension developers, especially those who are in fact exploring the space of what can be done with things outside the existing stable API.
I think the above list of problems we're trying to solve, and why Jetpack or some other stable API with growing surface area would help us solve them, is so much in the hindbrain of the people who've been banging their heads on this problem that it's easy to forget that none of the above is obvious "from the outside"....
To be clear, the stable API doesn't have to be Jetpack. Jetpack is the current attempt at creating such a thing, as I understand it.
January 11, 2010 06:32 AM
January 10, 2010
Mike Pinkerton -- I hate having to repeat myself
January 10, 2010 09:31 PM
Last updated: February 09, 2010 10:15 PM




