You are here

Home

Fun With Enforcers

Tue, 16/12/2014 - 23:33 — root

I need to mention a fun thing that I did almost exactly when this site went offline. I signed up to volunteer in an RCMP training exercise scheduled for June 18, 2013. At the information session, the head of Risk Management UBC told us that one could either get up at 8am to play a regular hostage or get up at 6am to play a hostage wearing bullet hole makeup. I chose the former.

RCMP officers at UBC.

The RCMP officers tried to keep their techniques as secret as possible. I got this picture before the hostages went to their assigned buildings, but I didn't take any once I was inside. I don't think anyone else did either, for fear of being kicked out. The closest thing we have to a picture of the actual rescue comes from an article in The Ubyssey.

When the event started, the group I was with went to Totem Park and the other group went to the "Force Sciences Building". People with Twitter accounts also posted that there was a shooter on the campus (with a disclaimer that the message was fake). I guess some sense of panic makes the situation more realistic for the cops. When we got to Totem Park, we met two friendly hostage takers from the LAPD. The RCMP wanted people from the outside to be posing as the shooters to make their actions harder to predict. The LAPD guys had shotguns loaded with blanks and they told us that the Emergency Response Team would be firing paintballs.

They also told us what it was like to work for a SWAT team. Some people were surprised to hear that they resolved situations 99% of the time without firing a shot. They said that one of the criminals from the other 1% was on a rampage under the influence of PCP. This went on for quite awhile, since the police first tried to negotiate with phone calls. The police outside were staying hidden in the bushes but we accidentally saw one of them because of the building's cleaning staff. They asked what the guy in the bushes was doing so they must've been unaware of all the emails about the training exercise.

The negotiation obviously reached an impasse since no one had millions of dollars to trade for hostages. When the stairwell door finally opened, at least 20 cops came running and there was no holding them back. The LAPD guys fired a few blanks into the hall, but they were soon pelted by paintballs and handcuffed. We were all crouched behind chairs during the rescue, but none of the paintballs came close to hitting us. The cops were very accurate. After this, we had to follow a specific route outside and keep our hands visible to the cops at all times. You never know who might be an inside man. We can only speculate about how they figured out which floor we were on.

Something else I worked on in June 2013 was an automated solution to the problem of top posting in emails. This refers to the increasingly common practice of quoting everything a person just said and writing your reply to it on top. In a throwaway email that will only be read once, this is fine because it minimizes scrolling. But in an email of value that people might want to read repeatedly, top posting is rude. It forces the person to read backwards in order to ascertain the context of the post. There is a self-referential rant about top posting that people circulate a lot:

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

I've had to explain why I bottom post more than once because some people don't even realize that it is saving them time. Strictly speaking, it is not correct to post everything at the bottom either. RFC1855 recommends a summary at the top followed by specific replies below the relevant quotations. But replies that are all at the bottom still look much nicer than replies that are all at the top. Since the task of moving replies down is completely mechanical, I went looking for programs to do it. The only one I found is by Arthur Pemberton. Since this is all about making posts look nice, I have cleaned up the formatting below.

  1. #include <stdio.h>
  2.  
  3. #define MaxLine 5000 /* Maximum number of characters per line */
  4. #define MaxMesg 5000 /* Maximum number of lines per file */
  5. #define EOLN '\n' /* End of line character */
  6.  
  7. int interspersed; /* indicator to say whether the message is interspersed or not */
  8. int NumberOfLines; /* Counts the total number of lines in a message */
  9.  
  10. struct MessageStruct {
  11. 	int quoteorder; /* The number of leading quote symbols in this line */
  12. 	int linenumber; /* The initial number of this line */
  13. 	char data[MaxLine]; /* The line string */
  14. } line[MaxMesg]; /* An array of lines, ie. the whole message buffer */
  15.  
  16. void ReadTheMessage()
  17. {
  18. 	char c;
  19. 	int n = 0; /* Current line counter */
  20. 	int i = 0; /* Current character counter */
  21. 	int endoffile = 0; /* End of file indicator */
  22.  
  23. 	NumberOfLines = 0; /* Message is empty at this point */
  24. 	while ((!endoffile) && ((c = getchar()) != EOF)) /* Read in the data from stdin */
  25. 	{
  26. 		/* Initialize the quote order. It will be counted later... */
  27. 		line[n].quoteorder = 0;
  28. 		i = 0; /* Reset the character counter */
  29. 		while ((c != EOLN) && (c != EOF)) /* Read in the current line */
  30. 		{
  31. 			line[n].data[i] = c; /* Feed the message buffer with the data */
  32. 			i++; /* Move to the next character */
  33. 			c = getchar(); /* Read it, and repeat */
  34. 		}
  35. 		line[n].data[i] = EOLN; /* Finish the string */
  36. 		line[n].linenumber = n; /* Set the line number */
  37. 		n++; /* Move to the next line, and repeat */
  38. 		if(c == EOF) endoffile = 1; /* Handle the premature end of file */
  39. 	}
  40. 	NumberOfLines = n; /* Save the number of lines and finish */
  41. }
  42.  
  43. void WriteTheMessage()
  44. {
  45. 	int n = 0; /* Current line counter */
  46. 	int i = 0; /* Current character counter */
  47.  
  48. 	while (n < NumberOfLines) /* Write the data to stdout */
  49. 	{
  50. 		i = 0; /* Reset the character counter */
  51. 		while (line[n].data[i] != EOLN)
  52. 			putchar(line[n].data[i++]); /* Write the current line */
  53. 		putchar(EOLN); /* Write the end of line */
  54. 		n++; /* Move to the next line, and repeat */
  55. 	}
  56. }
  57.  
  58. void AnalyzeTheMessage()
  59. {
  60. 	char c;
  61. 	int n = 0; /* Current line counter */
  62. 	int i = 0; /* Current character counter */
  63. 	int k = 0; /* Current quote order counter */
  64. 	int orderlessthenk = 0; /* Indicator if there is a quoteorder less then k */
  65. 	int maxquoteorder = 0; /* Maximum value for the quoteorder */
  66.  
  67. 	/* Ok. First we count the so called "quote order" of each line,
  68. 	 * which amounts to number of leading '>'s per line. In the process,
  69. 	 * we need to account for the famous "From" bug in sendmail, and
  70. 	 * decrement the quoteorder if the text starts with the word
  71. 	 * "From", in order to get it right.
  72. 	 */
  73. 	for (n = 0; n < NumberOfLines; n++) /* Go through the each line */
  74. 	{
  75. 		i = 0; /* Reset the character counter */
  76. 		c = line[n].data[i]; /* Set the initial data */
  77. 		/* Count the leading '>'s, skip spaces */
  78. 		while ( ((c == '>') || (c == ' ')) && (c != EOLN) )
  79. 		{
  80. 			/* Increment quoteorder for each '>' */
  81. 			if (c == '>') line[n].quoteorder++;
  82. 			i++; /* Go to the next character */
  83. 			c = line[n].data[i]; /* Read it, and repeat */
  84. 		}
  85. 		if ( (c == 'F') && /* Check if the following string is "From" */
  86. 		     (line[n].data[i + 1] == 'r') && /* Now, this is very, very ugly... */
  87. 		     (line[n].data[i + 2] == 'o') &&
  88. 		     (line[n].data[i + 3] == 'm') ) line[n].quoteorder--;
  89. 		/* If yes, decrease the quoteorder... */
  90. 	}
  91.  
  92. 	/* Now we need to check if the message is interspersed or not. This is tricky,
  93. 	 * as we need to find constructions of the form:
  94. 	 *
  95. 	 * > > >
  96. 	 * >
  97. 	 * > > >
  98. 	 *
  99. 	 * and similar, which amount to quoteorder being 3,1,3 and the like.
  100. 	 * Simoultaneously, we *don't* want to find constrctions of the form:
  101. 	 *
  102. 	 * >
  103. 	 * > > >
  104. 	 * >
  105. 	 * 
  106. 	 * because this can happen even if the message is not interspersed.
  107. 	 * But that's life... ;-)
  108. 	 */
  109.  
  110. 	for (n = 0; n < NumberOfLines; n++) /* First find the biggest quote order */
  111. 		if (line[n].quoteorder > maxquoteorder)
  112. 			maxquoteorder = line[n].quoteorder;
  113.  
  114. 	interspersed = 0; /* Assume message is not interspersed */
  115.  
  116. 	/* We need to check the above construction for all nonzero quote orders */
  117. 	for (k = 1; k < maxquoteorder; k++)
  118. 	{
  119. 		n=0; /* Start from the begining of the message */
  120. 		/* Get to the block of lines of quoteorder k */
  121. 		while ((line[n].quoteorder != k) && (n < NumberOfLines)) n++;
  122. 		/* Go through the block */
  123. 		while ((line[n].quoteorder == k) && (n < NumberOfLines)) n++;
  124. 		/* Now go down the rest ot the lines, and check if some line has
  125. 		 * quoteorder less then k, and another line after it the quoteorder
  126. 		 * equal to k again, which amounts to interspersed construction.
  127. 		 */
  128. 		orderlessthenk = 0; /* Assume there are no quotes of order less then k */
  129. 		while (n < NumberOfLines) /* For each of the remaining lines */
  130. 		{
  131. 			/* alert if there is a quote of lesser order */
  132. 			if (line[n].quoteorder < k) orderlessthenk = 1;
  133. 			/* We found the interspersed construction!! */
  134. 			if ((line[n].quoteorder == k) && (orderlessthenk)) interspersed = 1;
  135. 			n++; /* Go check the next line */
  136. 		}
  137. 	}
  138. 	/* And that's it. If the message has interspersed structure on any 
  139. 	 * level of quoteorder, we have the indicator on. Otherwise, it is off.
  140. 	 */
  141. }
  142.  
  143. void SwitchLines(int i, int j)
  144. {
  145. 	int n;
  146. 	int tempquoteorder;
  147. 	int templinenumber;
  148. 	char tempdata[MaxLine];
  149.  
  150. 	/* This switches line i with line j, as needed for sorting (sigh)... */
  151. 	n = 0; /* Copy line i to temp */
  152. 	tempquoteorder = line[i].quoteorder;
  153. 	templinenumber = line[i].linenumber;
  154. 	while (line[i].data[n] != EOLN)
  155. 		tempdata[n] = line[i].data[n++];
  156. 	tempdata[n] = EOLN;
  157. 	n = 0; /* Copy line j to line i */
  158. 	line[i].quoteorder = line[j].quoteorder;
  159. 	line[i].linenumber = line[j].linenumber;
  160. 	while (line[j].data[n] != EOLN)
  161. 		line[i].data[n] = line[j].data[n++];
  162. 	line[i].data[n] = EOLN;
  163. 	n = 0; /* Copy temp to line j */
  164. 	line[j].quoteorder = tempquoteorder;
  165. 	line[j].linenumber = templinenumber;
  166. 	while (tempdata[n] != EOLN)
  167. 		line[j].data[n] = tempdata[n++];
  168. 	line[j].data[n] = EOLN;
  169. }
  170.  
  171. void SortTheMessage()
  172. {
  173. 	int i,j;
  174.  
  175. 	/* In order to make the message bottom-posted, we simply need to sort
  176. 	 * the lines by quoteorder descending. Further, we need to preserve
  177. 	 * the order of lines of the same quoteorder, which amounts in sorting
  178. 	 * same-quoteorder lines by line number, ascending. N.B.: Maybe this can
  179. 	 * be done in a more clever way, but I am lazy to ponder over it... :-)
  180. 	 */
  181.  
  182. 	/* The algorithm is just a simple, stupid and slow bubble-sort, over
  183. 	 * quoteorder and linenumber simultaneously... */
  184. 	for (i = 0; i < NumberOfLines; i++)
  185. 	{
  186. 		for (j = 0; j <= i; j++)
  187. 		{
  188. 			if (line[i].quoteorder > line[j].quoteorder) SwitchLines(i, j);
  189. 			if ((line[i].quoteorder == line[j].quoteorder) &&
  190. 			    (line[i].linenumber < line[j].linenumber)) SwitchLines(i, j);
  191. 		}
  192. 	}
  193. }
  194.  
  195. int main() /* This is self-explanatory, I hope... :-) */
  196. {
  197. 	ReadTheMessage();
  198. 	AnalyzeTheMessage();
  199. 	if (!interspersed) SortTheMessage();
  200. 	WriteTheMessage();
  201. }

The problem with this program is that it only makes an email look nice if you copy the email into a text file and then run the program on it. I wanted something with seamless integration into an email client. So naturally I set out to make a Thunderbird extension. The first lesson that became painfully clear is that Javascript is slow, even if you avoid jQuery. Sorting replies involves some very standard string manipulation but everything had to be done with specialized functions. Looping through a string (even a short string with only 1000 or so characters) gave this familiar message:

Dialog warning the user of a slow script in Mozilla Thunderbird.

I tried to figure out how long a message has to be before that warning is triggered but got very weird results. I had some emails that processed on the first try without timing out. When I added 10 characters to them, they didn't just show the warning once, they showed it again after I clicked "Continue". Messages can either be shorter than something that Thunderbird finds annoying or twice as long as something that Thunderbird finds annoying. I have no idea why.

Doing this with real emails instead of test strings also proved a bit cumbersome. Web browsers all follow the same Document Object Model which makes it easy to manipulate a webpage. To get to the body of a webpage, you simly call document.getElementsByTagName("body"); If you do the same thing in Thunderbird however, you run into a lot of unexpected elements because there is no standard for how to display emails instead of web pages in a web browser layout engine. The messages you could possibly view along with parts of Thunderbird's own interface are stored in an object hierarchy that Mozilla just made up. This took some trial and error but I eventually figured out that the equivalent method to call is document.getElementById("messagepane").contentDocument.getElementsByTagName("body");

Accessing the body this way, it became clear that it was laid out using blockquotes. Messages sent by another Thunderbird user are wrapped in <blockquote type="cite">. Traditional, text-based email clients signify quoting with the ">" symbol which the short program above is designed to use. For the most part, Thunderbird catches these and turns them into blockquotes as well. However, apart from the Mozilla blockquote, there is another type of blockquote that must be moved around: <blockquote class="gmail_quote">. Eventually I decided to just turn all Gmail blockquotes into Mozilla blockquotes as well to keep the style uniform. After working on this and a few other nitpicks, I released the extension as Posting Style Enforcer in case people want to blame the program for policing their posts. For people who have opinions opposite to mine, I included an option that reverses the sorting order and enforces top posting.

There is one type of email that will definitely not work with my addon - something sent from Hotmail. Hotmail does not use blockquotes, ">" or any other symbols for quoting. It simply sticks a bunch of blank lines underneath the new message and appends the old one. Separating paragraphs with multiple blank lines is something people do from time to time even when they aren't quoting anything so there is not much I can do about this. Perhaps it is clear in a discussion with only Hotmail users but Thunderbird users who want proper quoting already know that Hotmail sucks.

Tags: 
devel
example code
old news

Valid CSS and XHTML | Released under a Creative Commons License | Mobile version