Writings from the sticks

Got a curious question the other day. One I had never really thought much about but after a discussion it became clear to me why the questions were the way they were.

Bit flipping and bit masking is not intuitive.

And unless you’ve had to do it then approaching it for the first time may no make a pile of sense.

For instance, if you have a look at the binary representation of a value in a Uint8, with code like

Dim i As UInt8 = 8
Dim s As String = Bin(i)

break

that 8, in binary, is 1000 (note I consider it a bug that any leading 0’s are not shown – they should be as the value 1000 is only 4 bits not all 8. Xojo’s Bin and ToBinary do not show it which is, incorrect, IMHO)

A Uint8 is 8 bits wide/ a Unit16 is 16 bits wide. A Uint32 is 32 bits wide. And any value stored in there uses ALL those bits either set to 0 or 1. So a 32 bit version of 8 should be

&b00000000000000000000000000001000

With this starting bit of info out of the way bit masking using the built in operators, AND, OR, NOT and XOR becomes somewhat simpler. Especially if you write things in binary using the &b notation although with really big values it becomes hard to read. This is often why you see people use HEX – or hexadecimal. It’s more compact. Octal is another form that is less often used.

If we start with our original value, 8 , as a Uint8 (or &b00001000) and want to test if certain bits are set then the AND operator is VERY useful.

In binary the LOW BIT, or bit 0, is the RIGHT MOST one – just like in decimal the ones is right most, tens next to that and so on.

So if we want to test if bit 3 is set in &b00001000 we can do

dim result as integer = 8 AND &b00001000
if result <> 0 then
  // tada ! bit 3 IS set !!!!
end if

The AND operator will take two bit patterns and match them up. Where there is a 1 in both operands, 8 and &b00001000, the result will have a 1. Since 8 is, in binary &b00001000 we get a result of &b00001000 – bit 3 is set in both 8 and so the result gets a 1 in that bit position.

     8 => &b00001000
AND       &b00001000
---------------------
          &b00001000

If we try a different value, 9, then we get results like

     9 => &b00001001
AND       &b00001000
---------------------
          &b00001000

Note the representation of 9 is basically “8 + 1”. Again when we apply the AND operator we get a result that is not 0. From that we know that in the two patterns some bits matched and gave us a result that was NOT all 0’s.

Other operators do slightly different things.

The OR operator will make sure that bits in the result are set to 1 if a bit is set to 1 in either operand. if we wanted to make sure that bit 3 WAS set then we could use OR

     8 => &b00001000       0 => &b00000000
OR        &b00001000    OR      &b00001000
---------------------   ------------------
result    &b00001000    result  &b00001000

In the left example above we started with the bit set. OR will not change it and so it remains set in the result. In the right hand example we start with the bit NOT set. OR makes sure it is set and the result has the bit set.

The NOT operator will flip all bits in the result. It only takes 1 operand. Any that are set to 1 will be flipped to be 0 in the result. Any that are 0 will be flipped to 1.

     8 => &b00001000       0 => &b00000000      170 => &b10101010
NOT                     NOT                    NOT
---------------------   ------------------     ------------------
result    &b11110111    result  &b11111111     result  &b01010101

In the left most example above we started with the bit set. NOT will change it and so is cleared result. But not also inverts all the other 0’s to 1’s as shown in the middle example. In the right hand example we start with the bit pattern for 170 (&b10101010). When we use NOT it flips every bit to the opposite.

XOR is the weirdest one. it full name is “exclusive or”. What that means it in its 2 input operands the result will be a 0 every where the inputs match. And a 1 everywhere they do not.

     8 => &b00001000        0 => &b00000000         170 => &b10101010
XOR       &b10101010    XOR      &b10101010     XOR        &b10101010
---------------------   -------------------     ---------------------
result    &b10100010    result   &b10101010     result     &b00000000

In the left most example you can see that we get 1’s in every spot where the second operand has a 1 and the first doesnt. But because bit 3 matches we get a 0 instead of a 1. The middle example shows that when one operand is all 0 you get a copy of the second operand. And in the right most we go no 1’s because all the spots in both match in every position and so we get all 0’s.

As I said writing these out for anything larger than an 8 bit value as binary is tedious and error prone. And so you often see hexadecimal used.

So how to determine the “right” hex to use ?

Again, EVERY integer is ALWAYS as the FULL width of whatever type you used. So always make sure you write things out in full.

We’ll start simple with an 8 bit value.

To convert &b10101010

• write it in groups of 4 bits each(&b1010 1010)
• each group of 4 bits is ONE hexadecimal “character”
• the low bit position in the group of 4 is like the “ones” in decimal, the next like the tens and so on.
• the lowest bit, if set, means a value of 1
• the next bit, if set, means a value of 2
• the next bit, if set, means a value of 4
• the next bit. if set, means a value of 8

so in our case, &b1010, means, working from right to left, 0 + 2 + 0 + 8 or 10 * (You can check thins in Xojo code if you care to)

Now the screwy part. Our normal counting system is base 10. so whenever we get to 10 we start a new position to the left of the current one. In hexadecimal you count up to 16 – not 10. And so up to 9 things are the same but then for 10-15 we use the letters A to F (A = 10, B = 11, C = 12, D = 13, E = 14, F = 15) You convert each group of 4 bits exactly the same way.

So in hexadecimal our value is &hAA (= &b1010 1010) Again you can check this in code

if &hAA = &b10101010 then
  msgbox "Norm was right"
end if

Because hex is much more compact its widely used. And for large values its simpler to read and can be used just as effectively for bit masking. You just have to know how to convert you binary bit masks into hex.

*The number theory behind how decimal, binary, hexadecimal, octal and other counting systems works is really quite interesting. They’re all positional based systems and the difference is the base they use, not so much HOW they use the digits in the specific counting system. Heck we all use one every day that is base 60 and it seems perfectly normal whenever the minutes go by and turn into hours 😛

Lately I’d been hanging out on some alternative Xojo discussion venues.

The one I’m finding useful is IfNotNil as they permit discussions of ANY topics which lead to cross platform development. So you can talk about Rust, Go, B4X products, LiveCode, Web and other toolsets you might use for cross platform software development.

Most of the time when we talk about events the first thing that pops into someones head isa UI control and the events is has for interaction with the user. A pushbuttons Action (or Pressed) event for instance. Or the various events on listboxes for selecting rows, handling mouse clicks, keyboard events and the myriad of other user interface and interaction related things listboxes handle.

But thats not the only way to use events 🙂

Sometimes you may write a control and it needs to get data from the layout its on or from something else outside its control. So how do you get access to this ?

You could, as I’ve seen in some code, make everything PUBLIC and then the specific control just gets its parent control and grabs what ever property or calls whatever parent method it needs.

This is, IMHO, a bad practice as it makes your code more fragile since a change in the method or property you’re accessing this way could cause unrelated code to break. And because it destroys any encapsulation you might have created. Its a huge violation of the Law Of Demeter I wrote about before. You should only use things that are passed to you as parameters, that you own and control, or that are declared in your code. Reaching way outside to the parent control would definitely violate this principle.

So what can you do ?

As discussed before, subclasses CAN create NEW events and you could use an event to ask the containing layout for whatever property value or reference to something else the layout has access to.

We’ll create a small project to demonstrate the idea. What we will have is a custom canvas control that will ask the outside world for the color that it should draw itself as. You might normally do this as a simple property but I am NOT going do that to demonstrate. The other thing we will have on the layout is yet another custom canvas that acts simply as a color picker.

I’m using 2019r1.1 so some things may be different if your using a newer version

First start a new desktop project

Add a new subclass of Canvas by dragging the canvas control from the Library over to the Navigator (or you can add a new class and set its super to Canvas in the inspector). Name this new class ColorSelectorSwatch.

Add a public property called FillColor as Color

We’ll add the MouseUp event handler and put the following code in it

  Dim c As Color
  
  If SelectColor(c, "Select a color") Then
    Me.FillColor = c
    Me.Invalidate
    RaiseEvent Updated
  End If

Also add the Mouse Down event handler and put this code in it

  Return True

Also add a new Event Definition and name it Updated

Thats it for our little color selector swatch

Add an instance of our ColorselectorSwatch to the default layout, Window1, by dragging it from the Navigator on to the layout Window1. By default its name will be ColorSelectorSwatch1

Add the event handler for the new Updated event. We’re not going to put any code in it just yet. We’ll return to this later once we create our other custom control.

Now add another canvas subclass to your project. Again you can do this by dragging or by simply adding a class and changing the superclass in the inspector. Name this class CustomCanvas. This is the one we’re going to add the event to that will ask the outside world what color it should draw itself in.

Add the Event handler for the Paint event to the CustomCanvas.

Also add a new event definition colorToDrawWith() As color.

Note that this event handler has a return value that is a color. When we need to get the color we’ll raise this event, get whatever return value, and then paint our canvas with this color. *

The code for the Paint event is fairly simple

Dim c As Color

c = RaiseEvent ColorToDrawWith // call the event to get the color 
                               // from the outside world

g.ForeColor = c
g.FillRect 0,0, g.width, g.height

Drag an instance of this custom canvas onto the default window and make sure it does not overlap your other canvas. By default it will be named CustomCanvas1.

Now we’ll add the handler for the ColorToDrawWith event. All we need is one line

return ColorSelectorSwatch1.FillColor

Now return to the Updated event of ColorSelectorSwatch1. In there we need one line to make it so our CustomCanvas will know to redraw itself when the color selector swatch gets a new color selected. In the Updated event put

CustomCanvas1.Invalidate

And run. Every time you select a new color in the color swatch the other canvas will update and redraw itself. And, to do so we have used an event to return information to the control instance that tells it what color to draw with.

Here’s my completed sample

*Of note one of the things that got added in versions after 2019r1.1 is a way to tell IF the event is implemented. You could use this to know whether you should redraw with the color returned or not.

The only change would be to make the Paint event of CustomCanvas read as follows

If IsEventImplemented("ColorToDrawWith") Then
  Dim c As Color 
  
  c = RaiseEvent colorToDrawWith
  
  g.ForeColor = c
  g.FillRect 0,0, g.width, g.height
End If

Xojo now generally uses exceptions to handle errors.

And with that comes some things you should know about how exceptions work that you need to be aware of before you cause yourself problems.

First off, in my opinion, you should put your try catch blocks as close to the code that causes the error. While Xojo lets you use one EXCEPTION or CATCH block at the end of any method like the following

// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 
// do a bunch of code 


Catch foo As RuntimeException // or Exception foo as RuntimeException
Break

I would suggest you avoid this form. It’s not obvious unless your methods are VERY short and it on one screen of code. For instance, does the following method have a catch ? Without actually looking at the very end you cannot tell (In fact it does but that isnt clear)

As well using a single consistent form is probably a better practice than sometimes using this form and sometimes using other forms. Even if the entire method is surrounded in a TRY CATCH block its more obvious. This is the exact same method with the addition of the TRY keyword at the very beginning and “END TRY” at the very end

try
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  // do a bunch of code 
  
  
Catch foo As RuntimeException 
    Break
End Try

The other thing to be careful about is how you write your catch statements. The documentation shows a legal CATCH statement as

Catch [[ErrorParameter] [As ErrorType]]

Note that EVERYTHING except the CATCH keyword is optional. Again I would suggest you almost NEVER use just a CATCH by itself as you will catch every type of exception, perhaps even those you cannot actually handle.

The other thing that is often confusing is if you write

Catch someException

which is legal what you have in fact got is

Catch someException as RuntimeException

and the local variable is name someException. Save yourself the grief and ALWAYS use the fully spelled out form of CATCH like

Catch <localVariableUsableInTheExceptionHandlingCode> as <ExceptionType>

Using it this way insures you are never surprised at catching more than you were prepared to handle. Also note that the localVariableUsableInTheExceptionHandlingCode is in fact defining a local variable that you can use in the body of this specific CATCH block. What that means is code like

Catch iox as IOException
   system.debuglog iox.errormessage

is legal and, in this case, iox is usable within the rest of the code following that catch until you get to another catch, the finally block, or the end try. The following WOULD NOT be legal

Catch iox as IOException
   system.debuglog iox.errormessage
Catch oob as OutOfBoundsException
   system.debuglog iox.errormessage

iox would not be in scope in the code following the catch of the out of bounds exception. Nor would oob be usable in the block handling the IOException.

And this last example shows something you can legally do. You can have more than one catch statement. The only thing you need to be careful about here is not catching the most general types before the more specific ones. What do I mean by that ?

Since all exceptions are descendants of RuntimeException they are more specific types of exceptions than RuntimeException is. If your code was like

Try
  Raise New nilobjectexception
  
Catch rte As RuntimeException
  Break
Catch noe As NilObjectException
  Break
End Try

you would find that any code following the catch of the NilObjectException would NOT get executed. Because NilObjectException is descended from RuntimeException the first catch block will handle EVERY exception. Changing this code to

Try
  Raise New nilobjectexception
  
Catch noe As NilObjectException
  Break
Catch rte As RuntimeException
  Break
End Try

will now execute the NilObjectException code since that matches the more specific type. Note this is not peculiar to try catch statements but a general thing to keep in mind when dealing with if then else, select case and other kinds of statements that use boolean values to determine if their code should execute.

Now go catch stuff 🙂

Bit shifting can be troublesome. Especially if you’re not sure WHY they can be troublesome.

So some recommend only using UNSIGNED integers for such operations. And using *2 and \2 to do bit shifting.

At a very low level most modern CPUs implement certain numerical standards in various instructions. And, of note, nearly every last one has two (or more) forms of shifts. One is a logical shift, and one is an arithmetic shift.

The difference between them is VERY subtle – but important.

It turns out that this is the reason that you SHOULD use unsigned integers IF you use * 2 and \ 2 to do bit shifting. When you multiply by 2 a modern compiler might realize this is a simple arithmetic shift and write the machine code to perform exactly that operation. An arithmetic shift will preserve the sign. And so when you do a \2 operation you do not get a result that flips signs. The sign bit (left most bit) is preserved. For instance

Dim i as int8 = -8 // this is &b11111000 in binary 
                   // MSB is 1 indicating a negative value

dim j as int8 = i \ 2 
// j is now -4 which is &b11111100 in binary
// note the MSB is STILL set so the result did NOT change sign

Arithmetic shifts generally preserve the sign esp when the sign bit is initially set. Otherwise you get odd sign changes.

But the same is not true when you do a LOGICAL shift. This just moves bits without regard to signs. Starting with almost the same code, but using BITWISE.ShiftRight instead of \2 we get very different results IF you expect to get a value you can use as a number.

Dim i As Int8 = -8 // this is &b11111000 in binary 

Dim j As Int8 = BitWise.ShiftRight(i,1,8)
// j is now 124 which is &b01111100 in binary

The shift this time made NO effort to preserve the sign bit as we were doing logical shifts which make no effort to preserve the sign bit.

So when, or if, you need to do any bit manipulation make sure you understand whats going on at a low level and IF you use * and \ to do shifting use unsigned values.

Shifts away !

Had an interesting discussion the other day and in the course of the discussion the use of prepared statements came up. Turned out that a few people had never used them and did all the leg work to created their own mechanism to concatenate strings.

For a long time in REALbasic. and REALStudio this was a requirement. Prepared statements didnt exist. But once they did I always recommended people use them instead of trying to roll their own set up.

Why? What are the advantages to using prepared statements instead of a roll your own set up using string concatenation?

First off security is a BIG reason to use them. SQL injection attacks used to be common simply because there were so many ways to cause them and trying to manually handle every case, even for a single database, could be difficult. There maybe cases where control characters are allowed and others where they are not and knowing the difference in any code that applied replacement patterns would grow into a very large body of code to deal with.

With string concatenation you run the risk of someone crafting an input string just so and wiping out your database. Hackers are very clever & skilled at this. So when a database has a mechanism designed to thwart them, like prepared statements, we should use it.

Specifically suppose you just made sure that you wrapped user input in quotes and passed that along.

   dim sql as string = "select * from table where column = " + WrapInQuotes(userInput)

and all WrapInQuotes did was

Function WrapInQuotes(inputString) as string
  // this is deliberately VERY naive to illustrate the point
  return """" + inputString + """"
End Function

This is deliberately VERY naive but it shows the reason prepared statements are used 

Some database will let you execute more than one statement as part of their API – which Xojo uses. If a “hacker” were to put in, as user input,

      "; delete from table;

and you used the above code you would find that the sql end up as

    select * from table where column = ""; delete from table;"

and you just might find your table no longer has data in it. And despite your and my cleverness there are so many screwy little edge cases that its almost impossible to get any kind of code with replaceall to work right in all cases.

Prepared statements avoid all this as the statement doesnt get parsed in the same way as a full sql statement that has all the values stated. The parameters are basically marked as “not present” but they know the type, position (or a name) and the execution of the statement then passes those values to the DB engine via its API. What it does NOT do is reparse the sql every time and it also makes no attempt to “interpret” the data values as if they were part of the commands.

They are JUST data and cannot be part of a command – and so the engine doesnt try to use them as commands which it does do in our previous example because it has to understand the statement(s) in order to execute them

The other upside to prepared statements in many databases is that they can do the leg work to create the query plan for the specific statement once and then reuse it over & over as you pass new variables. Not all databases can do this when using the various Xojo API’s but some do.

So there can be speed benefits to using them as well

But, lets be careful out there

Xojo’s BIN, HEX and OCT methods and their new counterparts, ToBinary, ToHex, and ToOctal should do a better job than they do.

If you pass in an 8 bit value you _may_ get an 8 bit binary representation back – or not. And this is true for ALL numeric types. The string you get back may not represent the full value esp if the upper bits are 0. This can be a problem sometimes especially if you’re trying to debug an algorithm that flips bits and you want to see all of them in a string representation (see this bug report)

If you’re trying to use the newer API 2 style then you’ll fnd that single and double values have no ToBinary, ToHex, or ToOctal extensions (see this bug report )

And BIN, HEX and OCT do not work on currency values and it also has no extension methods (see this bug report)

So to fill this gap and fix these shortcomings I have created a module that you can add to your projects that will give you the proper representations AND that will allow you to convert a currency to binary, hex or octal and then convert that back to a currency.

Enjoy !

There aren’t a lot of items in a Xojo desktop project that you can’t rename. You can change the name of the App instance, the default window, the default menu bar and many many others.

But there are a handful that, if you do rename them, you may inadvertently break your application. This is entirely due to how the Xojo framework works, and in part how macOS itself works.

One group of items you should not rename is several Edit Menu items. Do not rename EditCut, EditPaste, EditClear, and EditSelectAll. The reason is, that if you do, then certain classes in the Xojo framework will no longer handle those menu items by default.

The simplest demo of this is to

• start a new desktop project
• add a TextArea to the default window – Window1
• run

Now if you type in text to the text area and press the shortcut for Select All (⌘-A or Ctrl-A) the text area will automatically handle selecting all the text.

Quit your little demo app and in the menu editor rename EditSelectAll to MyEditSelectAll.

Run the small sample again. Perform the same test by adding a bunch of text and trying to use the shortcut. This time the menu shortcut is NOT automatically handled and likely just beeps at you.

Several controls have behaviour that is pre-configured to rely on the Edit Menu items being named in a specific way. I’ve written about how you can do this as well before. Renaming the Edit Menu items then breaks that automatic configuration.

The other thing that renaming is specific to macOS. On macOS the OS will automatically add new menu items, the ones for Start Dictation and Emoji & Symbols, to the edit menu IF it the TEXT of the for the Edit Menu title is “Edit”.

If you alter it to anything else then macOS will not find the correct menu to insert these items and they will not be added to your Edit Menu.

Again this is easy to test with the tiny sample app we have so far. Just change the Edit menus text to Edit followed by a space and run. When you click on the Edit menu you will not longer have the 2 automatically added items.

This looks like it could be a nice forum as a place to learn about and discuss cross-platform app development, not specifically focused on Xojo (although that will be one of the topics I’m sure)

As an alternative to the Xojo forums where you cannot discuss other products etc this one is focused on cross platform – regardless of toolset

I’ll give it a chance. You should as well.

A small handful of changes in 2019r3.1

Module Color
alpha changed to Integer
Function CMY(cyan As Double, magenta As Double, yellow As Double, alpha As Integer = 0) As Color

ColorGroup
gained operator_compare
Function Operator_Compare(otherColor As Color) As Integer

Module Crypto
added PBKDF2 with HashAlgorithms parameter
Function PBKDF2(salt As String, data As MemoryBlock, iterations As Integer, desiredHashLength As Integer, hashAlgorithm As HashAlgorithms) As MemoryBlock

Class DatabaseColumn
now has a destructor
Sub Destructor()

Class DatabaseRow
added Column setter method
Sub Column(name As String, Assigns value As Variant)

added Destructor
Sub Destructor()

Module String
new overloads for IndexOf
Function IndexOf(Extends s As String, startPosition As Integer, searchString As String, options As ComparisonOptions = ComparisonOptions.CaseInsensitive, locale As Locale = Nil) As Integer

Function IndexOf(Extends s As String, searchString As String, options As ComparisonOptions = ComparisonOptions.CaseInsensitive, locale As Locale = Nil) As Integer

new method LastField
Function LastField(Extends s As String, separator As String) As String

Class TextArea
changed type of SelectionAlignment type
Property SelectionAlignment As TextAlignments

Cant say that so far I’ve used it much.