Readability assessment 2019

The first task in a readability assessment is to establish two things:

1. What is the purpose of each letter – what does the Tribunal want the reader to do in response?
2. Who is the primary audience for the letter? Whom is it addressing?

Below are summaries of the purposes of each letter. We look at how effectively they present these purposes to the reader. We also flag areas where letters seem unclear about their audience.

Acknowledgement of complete notice of appeal - Appears to be late

This letter stems from a Notice of Appeal in the General Division - EI stream. It is the first point of contact between the Tribunal and an appellant whose Notice of Appeal appears to be late. The Tribunal sends the same letter to appellants who have supplied reasons for lateness and those who have not.

Audience

This letter has two audiences. It is addressed to both the appellant and the appellant’s representative. It starts out speaking directly to the representative, referring to the appellant in the third person. The voice suddenly shifts, after the subheading Additional Information, to the second person ‘you’. ‘You’ is intended to include both the applicant and the representative.

At this stage in the appeal process, according to data you have supplied me from the 2017/18 fiscal year, the great majority – 74% of EI appellants – are unrepresented. My recommendation is to speak directly to them throughout. This averts the danger that appellants, feeling that the letter is not really addressing them, will not read very far. That could cause them to miss information that they will need in future.

To cover off appellants who are represented, we should add an optional “Copy: Representative” line at the end of the letter.

Purpose

The letter has the following purposes:

1. To acknowledge that the Tribunal has received the Notice of Appeal and that it is ‘complete’ in the sense that the Notice of Appeal form is not missing any required information
2. To tell them that the Tribunal has to decide if the Notice of Appeal is really late or not, and if it really is late, whether or not to grant an extension
3. To confirm that the appellant has a representative and explain how the Tribunal will communicate with them from this point forward
4. To tell them to include the Tribunal’s file number on all documents they send
5. To tell them to keep all documents they receive and keep copies of what they send
6. To explain that they will receive copies of documents sent by other parties
7. To ask them to keep the documents confidential
8. To tell them to inform the Tribunal of any change in contact information
9. To tell them how to contact the Tribunal if they have questions

Stating the letter’s central purpose in the subject line

An initial heading or subject line is very useful in a letter because it can ‘telegraph’ the most crucial messaging in the letter. The subject line for this letter encapsulates its main purposes, but is not written in the active voice. In keeping with my recommendation to shift the voice to the second person, I suggest this main heading:

We have received your notice of appeal. The Tribunal must now decide if it is late.

Removing misleading purposes

The letter does not really have any action item for the appellant or their representative. They are not supposed to send any more information unless the Tribunal member requests it. The instructions in purposes 4 to 7 tell appellants and representatives how to do things you don’t want them to do yet. In literacy task analysis, that is what we call a ‘plausible distractor’¹.

I recommend removing the documentation instructions to keep the focus on the central purposes of this letter. Documentation instructions should be included in the letters that follow.

Removing distracting information

Plausible distractors are also at work in the detail given about the rules around late filing. As you have explained to me, a review for lateness is more complicated than it seems:

• The 30-day time limit is really 40 days because of an internally-allowed grace period.
• The member might decide it was not really filed late for a variety of reasons, such as a postal strike or CEIC mailing to the wrong address.
• The member might decide to grant an extension based on various reasons supplied by the applicant.
• If the Notice of Appeal was filed later than a year after receiving the reconsideration decision, the member cannot grant an extension.
• The file might go right to a Notice of Hearing where the lateness issue would be discussed.

In the letter, there is information about the 30-day limit, the one-year limit, and extensions. This is distracting because for any given appellant, some of the information will not berelevant to their individual situation. For example, the one-year limit would be irrelevant to those who filed well within that time period. Extensions would be irrelevant to those who may end up being “not really late” for reasons such as a postal strike.

All that appellants and representatives really need to know at this point is:

• They may have filed late
• A Tribunal member will decide whether or not this is the case
• Meanwhile, they should wait for further instructions.

The fact that the Notice of Appeal is “complete” is also not of interest to the appellant. This processing issue would only affect them if their Notice of Appeal were incomplete.

Using sub-headings to highlight other key messages

The other key messages in the letter would stand out better if they were set off with short sub-headings:

Purpose 3: From now on, we will write only to your representative

Purpose 8: Has your contact information changed?

Purpose 9: How to contact us

Appeal filed on time

This letter stems from a Notice of Appeal in the General Division – Income Security stream. The Tribunal sends this letter when a Member has reviewed the Notice of Appeal for lateness and decided that it was in fact filed on time.

Audience

The letter is addressed to both the appellant and the respondent – that is, Employment and Social Development Canada. It goes to the appellant’s representative if they have one, and directly to the appellant if unrepresented. All of the information in the letter applies to both parties.

There is only one inconsistency in the ‘voice’. The opening paragraph talks about “your Notice of Appeal” being filed on time, addressing the appellant. It would be better to say “this Notice of Appeal,” thus including ESDC in the conversation.

Purpose

The letter has the following purposes:

1. To tell the appellant and respondent that this appeal was not late and give the reasons why
2. To explain the Notice of Readiness process to both parties
3. To explain the options for next steps to both parties
4. To tell them to include the Tribunal’s file number on all documents they send
5. To tell them to keep all documents they receive and keep copies of what they send
6. To explain that they will receive copies of documents sent by other parties
7. To ask them to keep the documents confidential
8. To tell them to inform the Tribunal of any change in contact information
9. To tell them how to contact the Tribunal if they have questions

Stating the letter’s central purpose in the subject line

This letter’s subject line gives half of the central purpose – “Appeal Filed on Time.” However it does not signal to the reader that there is something they should do now, which is to take the next steps in the Notice of Readiness process.

A more complete subject heading would be:

The Tribunal decided this appeal was filed on time. This letter explains what to do next.

Using sub-headings to highlight key messages

Here are some examples of how to set off the other key messages with sub-headings:

Purposes 2, 3, 4, 5, 6, & 7 (This information has been re-organized so that the purposes appear in a more logical order):

Next step: Send the Tribunal documents to support your case by [Date].

How to send documents to the Tribunal

Receiving documents from the Tribunal

The Notice of Readiness form

What happens after you send the form

Purpose 8: Has your contact information changed?

Purpose 9: How to contact us

Notice of hearing - Teleconference

This letter goes to all parties involved in an appeal at the General Division level in both the EI and Income Security streams. It has the following purposes:

1. To inform all parties of the date and time of the hearing
2. To tell them how to get ready for the hearing and how to dial in
3. To explain why the hearing will be held by teleconference
4. To get them to contact you if they have witnesses
5. To get them to contact you if they need an interpreter
6. To get them to contact you if they need to change the hearing date
7. To tell them what happens if a party does not attend the hearing
8. To tell them to include the Tribunal’s file number on all documents they send
9. To tell them to keep all documents they receive and keep copies of what they send
10. To explain that they will receive copies of documents sent by other parties
11. To ask them to keep the documents confidential
12. To tell them to inform the Tribunal of any change in contact information
13. To tell them how to contact the Tribunal if they have questions

Stating the letter’s central purpose in the subject line

The Notice of Hearing includes both of its key purposes in its subject line – the format of the hearing and the hearing date. In the sentence style I am recommending for subject headings, this would be:

Your hearing will take place by teleconference on [date].

Using sub-headings to highlight key messages

There are a great many other key messages in this letter, and they are not always sequenced in a way that is logical from the reader’s point of view. Once they have been reorganized, they fall under these headings in this order:

Purpose 6: Contact us right away if you need to reschedule the hearing.

Purpose 2: How to dial into the teleconference

Purpose 3: Why is the hearing by teleconference?

Purpose 4: Do you have witnesses?

Purpose 5: Do you need an interpreter?

Purpose 7: What happens if one of the parties fails to attend the hearing?

Purpose 8: How to send documents to the Tribunal

Purposes 9, 10, and 11: Receiving documents from the Tribunal

Purpose 12: Has your contact information changed?

Purpose 13: How to contact us

You may decide to dispense with Purpose 3 if you find, after implementing the new directive to accommodate appellant’s preferences for their hearing format, that this issue no longer generates appeals.

Notice to potential added parties

The Tribunal sends this letter to employers who may have a direct interest in the outcome of EI appeal to the General Division.

The letter has the following purposes:

1. To inform the employer of the appeal and how they are connected to it
2. To explain the role of an added party
3. To tell the employer how to file a request to be added as a party
4. To tell them NOT to send any documents until a Tribunal member decides to add them as a party and they are notified in writing
5. To tell them to forward this letter to the correct person if they are not the intended recipient
6. To tell them to include the Tribunal’s file number on all documents they send
7. To tell them to keep all documents they receive and keep copies of what they send
8. To explain that they will receive copies of documents sent by other parties
9. To ask them to keep the documents confidential
10. To tell them to inform the Tribunal of any change in contact information
11. To tell them how to contact the Tribunal if they have questions

Stating the letter’s central purpose in the subject line

The subject line for this letter is “Notice to Potential Added Party.” This does not get at the action you are requiring of the reader. A more action-oriented subject line would be:

Do you wish to be added as a party to this appeal?

Removing plausible distractors

As with the letter acknowledging a late Notice of Appeal, this letter says a great deal about how to handle documentation that you do not in fact want the reader to send yet. This will lead to inappropriate actions and more time spent sorting them out on where they are in the process.

To keep the focus on this letter’s central purpose, I recommend sending information on documentation after the employer sends a request to be added as a party and a Tribunal member has accepted the request.

Using sub-headings to highlight other key messages

The remaining key messages would stand out better with these subheadings:

Purpose 2: The role of added parties

Purpose 3 & 4: We must receive your request to be added as a party by [Date] and What happens after you file your request

Purpose 10: Has your contact information changed?

Purpose 11: How to contact us

Potential added party – Documents filed

This Tribunal sends this letter to people and corporations who request to be added as parties to a General Division EI appeal. It is sent if they submit documents with their request, or before they have been approved as added parties.

It has these purposes:

1. To acknowledge receipt of the request to be added as a party
2. To explain the role of an added party
3. To acknowledge receipt of documents filed with the request
4. To tell them that, although the Tribunal secretariat will retain the documents, it can do nothing with them until they are approved as an added party
5. To tell them that furthermore, the Tribunal will not share documents from other parties with them until they are approved
6. To tell them to include the Tribunal’s file number on all documents they send
7. To tell them to keep all documents they receive and keep copies of what they send
8. To explain that they will receive copies of documents sent by other parties
9. To ask them to keep the documents confidential
10. To tell them to inform the Tribunal of any change in contact information
11. To tell them how to contact the Tribunal if they have questions

Stating the letter’s central purpose in the subject line

The subject line for this letter is “Potential Added Party – Documents filed.” This is a misnomer. These documents cannot be properly ‘filed’ with the Tribunal until the sender is approved as an added party. This subject line and the two paragraphs that follow could easily mislead a reader into concluding that they had already been added as a party.

A more effective subject line would be:

We are holding your documents until you are approved as an added party.

Removing plausible distractors

As with the Notice to Potential Added Parties we discussed above, this letter is full of plausible distractors on how to handle documentation that you do not in fact want them to send yet. This could easily lead to more inappropriately sent documents, more explanatory letters, more phone calls to sort things out, and so on.

To keep the focus on this letter’s central purpose, my recommendation once again is to refrain from sending instructions about documentation until the Tribunal member has accepted the request to be added as a party.

I also recommend deleting the information on the role of an added party. The recipient has already received that message in the Notice to Potential Added Parties and we know they understood it because they responded. There is no need to repeat it here, especially since we want to stress that they are not an added party yet.

Using sub-headings to highlight other key messages

Once we have removed unneeded or distracting information, the subheadings for the remaining key messages would be:

Purposes 4 & 5: We can take no action until you are approved as an added party

Purpose 10: Has your contact information changed?

Purpose 11: How to contact us

I think it is reasonable to expect that, if my recommendations for rewriting the Notice to Potential Added Parties are accepted, you will rarely have to use this follow-up letter at all.

I measured the Grade Reading Level of the sample letters using the Flesch-Kincaid Readability Formula⁵.

Document Name	Grade Reading Level Score
Acknowledgement of complete Notice of Appeal - Appears to be late	11.8
Appeal appeared to be late determined to be on time	11.6
Notice of hearing Teleconference	8.8
Potential Added Party - asking if they want to be a party	12
Potential added party – Documents filed (after they responded they wanted to be a party)	11
Average Grade Reading Level	11

Implications of these results

Reading level scores are a reliable predictor of reading difficulty based on syntactical density – the length of sentences and words.⁶ It is important to note that they do not analyze other cognitive barriers in the prose, such as confusing or disorganized information. Problems such as those noted in the Purpose section of this report can raise the level of reading difficulty much higher.

Our first two sample letters are more than four grade reading levels higher than our target. The Potential Added Party letters are closer to an acceptable grade reading level, provided that they go to an employer. I would caution you however that if similar language is used to contact added parties in the Income Security stream, the reading level is much too high.

The Teleconference letter is close to the high end of our Grade 6-8 target, but because it is such a crucial piece of information, we need to bring it down closer to Grade 6.

You have asked for my thoughts on problems you are encountering with appellants who receive their correspondence by email. Appellants are arriving at their hearings with their documentation incomplete, which causes adjournments. There are issues around appellants not opening their mail, not reading it, not printing it, and even deleting it, thinking it is spam.

Potential problems with the opting in process

In keeping with its Client Centric Action Plan to help speed up the appeal process, the Tribunal Secretariat has begun to encourage correspondence by email. In August 2018, a box was added to the Notice of Appeal asking permission to communicate by email.

Appellants who do not wish to receive correspondence by email can tick the ‘no’ box. But if an appellant writes to the Tribunal, even once, by email, all correspondence thereafter automatically comes by email. This is explained in the permission box on the Notice of Appeal, in very small type:

(If you do not authorize email communication, and then email us during the course of your appeal, we will continue to communicate with you by email.)

In addition to being in very small type, this warning tests at Grade Reading Level 13.6.

Part of the thinking behind Canada’s Anti-Spam Law (CASL) was that automatic opt-ins place an unfair burden on consumers to opt out.¹⁰ Although the Tribunal’s process is technically not an automatic opt-in, it is close enough, in my view, to cause confusion and to place a burden on unsophisticated email users.

Let’s take the case of our persona Richard, who has developed some mobility issues. Let’s say Richard filed his appeal in August 2018, when the email permission box was added to the Notice of Appeal. Richard ticked the ‘no’ box. He expected to receive his correspondence by mail. He did not read the warning very carefully, because the type was so small.

In the fall of 2018, Richard went to the library and wrote a quick email to notify the Secretariat of a change to his phone number. Then winter set in, and Richard’s trips to the library became less frequent. He completely missed this poorly worded and badly organized notification:

Veuillez ne pas répondre à ce courriel.

Merci d’avoir communiqué avec nous. Nous communiquerons avec vous si vous êtes dans l’attente d’une réponse.

Étant donné que vous avez communiqué avec nous par courriel, nous vous contacterons aussi par courriel à l’adresse que vous venez de nous fournir. La correspondance du Tribunal peut contenir des renseignements personnels. Veuillez-vous assurer de vérifier périodiquement si le Tribunal vous a envoyé des courriels ou nous informer de votre désir de ne pas recevoir de correspondance par courriel.

***

Please do not reply to this e-mail.

Thank you for communicating with us. If your request requires a response, we will contact you.

As you have contacted us by email, going forward we will correspond with you by email at the email address you have just provided. The Tribunal’s correspondence may include personal information. Please be sure to monitor regularly for email from the Tribunal or let us know if you do not wish to receive email correspondence. 

Because of a long stretch of bad weather, it took some time for Richard to return to the library and realize that he had been receiving his Tribunal correspondence by email. Now he is frantic because he has missed a crucial deadline in the appeal process.

Although this is only an imaginary scenario, I would suggest that Richard’s story or variations on it are quite plausible, especially among older and low-income appellants. Young Eric, who moved from Alberta back to the Maritimes, could easily have fallen into this trap as well, if he were initially receiving Tribunal communications by mail and then sent an address change notification by email.

My recommendation is that you review your process for ‘opting in’ to email. You need to ensure that you have a ‘double opt-in’, which marketers consider to be a best practice.¹¹ That means that recipients would have to confirm, after starting to communicate by email, that they wish to continue communication in this medium.

The cost of communicating by email

Agreeing to communicate with the Tribunal by email has consequences that are not made explicit in the opt-in process. Appellants may not realize at the outset that they will have to print a significant number of documents at their own expense, and that the documents they send will have to be scanned and attached.

For appellants trying to live on a very low income, such as Richard and Maria, the cost of printing and scanning documents could easily be prohibitive. Appellants need to understand the implications of using email before they make their choice, and they need to be reminded of these implications and their right to opt out of email regularly throughout the process.

The secretariat is moving toward a cover email instruction that advises email recipients to “open, print, and read” documents attached. This is a more concrete instruction than was on the original cover message, and will, we hope, lead to fewer parties having incomplete documentation at the time of their hearing.

However, appellants will find in their email attachments documents that they have already printed. They will see, for example, copies of documents they themselves have submitted. It would be unfortunate to make appellants feel they have to pay for printing a document twice.

Once solution to this might be to add a sentence to the cover email explaining that, if they already have a printed copy of an attached document, they should ensure that it has the document number on it for their reference. If it does not, they should open the attachment and copy the number from the attachment onto their printed copy.

Crafting effective subject lines for emails

The secretariat’s practice for email subject lines is in transition. To date, the practice has been to code the subject line with the file number, the name of the template attached, and the date of the correspondence. An example would be: GE-19-201 - Withdrawal of Appeal - March 19, 2019.

As we discussed at the beginning of this report, in any piece of correspondence, the purpose, or action item, should be paramount. The Secretariat is moving toward a new way of crafting email subject lines that focuses more on what is essential for the reader. For example:

Social Security Tribunal Appeal - NOTICE OF HEARING [Date of hearing]

I fully support this initiative and hope that the new subject lines I am writing for the letter templates will help direct staff writers to what they need to focus on in email subject lines.

File naming conventions

There seems to be some fluidity in the way that letter template files are named at the Secretariat. The file name used internally is currently modified before being sent out. It incorporates the subject line of the letter plus a document code if applicable. It seems to me overly complex to have two sets of file names – internal and external.

Furthermore, since I have extended letter subject lines into much more prominent headings expressed as sentences, transposing the letter subject line to the file name is no longer practical. The file name will have to be carefully crafted as a distillation of the subject line.

It is outside the scope of my contract to make recommendations on naming each file. I am however flagging this as an issue for the Secretariat. File names have to do two things:

• Signal something that makes sense to the external reader who will be opening the file as an email attachment
• Have a simple coding or numbering system that will enable internal users to easily retrieve and organize the template files. This coding should be removed when files are sent.