When we started 2023 Xojo had – 2013 open bug reports – 25516 closed bug reports – 4203 archived bug reports – 11.62 average daily # of new reports and the fist public case # I could find was 71329
As of Jan 1, 2024 there are – 2280 open bug reports (an increase of 267 – up 13%) – 26686 closed bug reports (an increase of 1170 – up 4%) – 4284 archived bug reports (an increase of 81 – up 2%) – 11.05 average daily # of new reports and the fist public case # I could find was 75266 (up 3937 – up 5%)
With each release through the year more reports and bugs were filed than fixed meaning Xojo slowly fell further & further behind fixing issues.
And some would have you believe that this is the end of C# on macOS.
Not true Unlike other proprietary tools that have a single IDE & compiler from one company C# has alternatives that can read & write VS projects and compile them on the Mac. It’s open sourced. Heck MS even made Roslyn, their compiler infrastructure open source.
A little over a year ago Xojo switched to using a new issue tracker IMHO this was a good move.
Not only does it seem to be more capable it lets them quit spending time on something they can buy and just use the heck out of. It lets them focus on their core product rather than ancillary things like bug trackers.
That said, a little after they did this switch I started watching some of the numbers that were publicly visible. Things like open case count, open bug count, etc.
A GIANT caveat. The numbers I see are NOT the same as what you see. From what I can figure out this is because it will depend on what cases are confidential and I might see a different set of those than you do. So the numbers should be close – but unless you’re me you will likely see different values.
That said here’s what I found :
The day I started tracking things there were 6313 open reports of all types (bugs, feature requests, etc), 29787 closed and a total of 36100 reports As of this writing there are 6960 open reports, 31924 closed and 38884 in total. 2784 new reports (note not necessarily BUGS – just reports) Thats an average of 7.6 new reports per day
Also the day I started there were 2018 open bug reports and 24769 closed. Today there are 2276 open and 26054 closed. Xojo has closed a lot of bug reports in the interim. But, as you can see, they are slowly falling behind. Perhaps why archiving reports was even started.
The day I started recording numbers the # of archived bug reports was 4154. Today that number is 4286 Note that it did creep up for some time but has halted since Xojo is no longer having their bot auto-archive old reports. So even after closing many reports, and archiving many, the # of open reports has still crept up quite steadily from release to release,
The other thing I have watched is the first public case #. This is the first report that I can see on any given day. It seems to be a purely sequential number applied to any report. That day the first # was 69400. Today its 73040. This suggests in total over the one year time frame there have been 3640 reports (feature requests, bugs, and whatever other categories) It also suggests there are many confidential reports that cant be seen. What type they are or what those might constitute I wont speculate about.
But it does suggest the average number of reports per day is some where between 10 and 11.
Having a years worth of data lends itself to some analysis. The average # of new reports (not specifically bugs) is about 11 per day. Of those, just under 50% end up being bug reports.
As well since we know the release date of every release during that period we can see how many reports, and bug reports there are from the last release to the next.
Release 2022r2 to 2022r3 (78 days) there were 868 reports (381 bug reports) 2022r3 to 2022r3.1 (6 days) there were 97 new reports (48 bug reports) 2022r3.1 to 2022r3.2 (19 days) there were 307 reports (130 bug reports) 2022r3.2 to 2022r4 (34 days) there were 475 reports (181 bug reports) 2022r4 to 2022r4.1 (5 days) there were 45 reports (28 bug reports) 2022r4.1 to 2023r1 (98 days) there were 983 new reports (318 bug reports) 2023r1 to 2023r1.1 (13 days) there were 196 new reports (85 bug reports)
As of today 2023r2 has not shipped. From the last release to now there have been 963 new reports & 334 bug reports.
Every release Xojo fixes many bugs, adds some new features, and makes other changes.
What their own issue tracking system says though is the the # of bugs fixed is quickly outpaced by the number of new bugs. You’ll notice on the graph that FIXED never exceeds BUGS which means there are more new bug reports than are being fixed.
So the number of open bug reports continues to grow over time. To the point they fall so far behind in fixing bugs that they archive bugs that they haven’t analyzed, reviewed, or fixed.
It seems clear from their own numbers that staying the course and doing the same things they’ve always done and in the same way they’ve always done is a losing proposition and eventually there will be so many open bugs that archiving simply has to start again. Or bug reports will simply be left to languish forever. Neither of these are great prospects.
A change in process that focuses on creating quality software from the outset, well before ANY code has been written, and extending to the point it’s released that includes extensive unit testing, REQUIRING unit tests for every piece of code, and expanding the unit tests as bugs are reported & fixed would be a great start.
Expanding the team so developers have the time to create all this is necessary. Not just nice, but necessary.
And someone needs to be “the code god” and have few development duties & more oversight and review every last check in and make sure they do meet all standards of code (format, unit tests, integration tests, etc)
And an expansion of the internal QA team is needed
The other thing I followed was the beta releases as far as possible. What I observed is that, for every beta release the # of participants starts out relatively high. If thread reads is an indicator of the actual # of downloads of a beta release. Since Xojo posts expiring download links it seems thread reads is a reasonable proxy for actual download. It should at least trend in the same direction as actual downloads.
Most betas seem to start with somewhere between 200 and 300 reads (downloads?) of the initial beta release. The one outlier here is Android which started with > 500 reads. However they all trail off and end up with somewhere around 150 or so.
There are supposedly hundreds of Testers yet the actual number that seem to test is a fraction of the total (about 1/6th)
Many testers haven’t even logged in during the latest betas. About 50% of all members of the testers group haven’t logged in since the beta for 2023r2 started.
The same trend was true for Android testing where about 20% of the entire testers group hadn’t logged in for the entire time Android was in beta (560 days)
Now you might ask yourself “if Norm can see this from outside surely someone at Xojo is watching these trends as well ?”
To be honest I have no idea if they are. Or arent. Either way the #’s done paint a picture of a process that is getting better, or of a very involved & engaged testing process.
However, there are things I am sure of. I’m sure this will get read. I’m sure I’ll get an email about it and how it violates some rule Xojo has never posted or has just invented for me. And nothing will change 🙁
But I have used it and in some cases its very handy since you may not be altering the menu item just changing the method that gets invoked.
For instance if you have a selection of numbers and want to invoke a “sort” method then you want something that will sort numerically. But if its a selection of non-numeric values then you want to sort in another fashion. Yet the menu item might just say “Sort”; or maybe sort ascending and one for sort descending. So the handler might be swapped out ahead of time as the selection is made depending on what sort of selection you have. And you can do the proactively instead of after the fact in the menu handler. This might not be that significant in some cases; in others it can be.
Use with care and consideration of the implications of ANY use of addhandler
Not that I do in every product I use for writing code. But its nice to have them. For instance in Xcode and VS Mac there are a TON of settings.
And I’ve customized very few. Why ? The defaults are decently chosen so I dont need to read them all understand them all and then set them.
BBEdit, perhaps my favourite text editor of all, is the same way
I _could_ customize the heck out of it but dont NEED to.
Now some might think “well if you never customize them why have them ?”. I said I never do. Thats not to say no one does. If they weren’t ever used or set they probably wouldn’t be there. But someone, somewhere, probably requested such a thing and these developers responded with a preference that lets people customize things to their liking rather than dictating “No you dont really need that do some thing else”
I’d love to have way more choices for things in Xojo as well.
And ones that provide additional customization like if I want a toggle button for line numbers which got removed and is now an IDE wide setting not a PER PROJECT one. Probably great if you only work on one project all the time but when you work on different client projects some like it on some like it off.