Sep 02

Audacity files are just XML documents, sooo…

I know it’s unlikely this particular solution—in which I use Audacity’s Labels Track interface to help me auto-generate HTML code I find tedious to write manually—will receive many “Wow! Just what I was looking for!” comments. It’s more about connecting dots. XML is like the universal translator on StarTrek, so if you have data in XML format it means you can easily get at that data and do anything else with it your heart desires. While I have access to leading proprietary audio applications that are appropriate for recording music, I use Audacity for all elearning narrations and voice-overs—the fact that it generates a valid XML document containing all the information about a project is one of the reasons.

I think the first time I intentionally opened a binary file with a text editor was probably using Notepad on Windows 3.0. “Gibberish” or “garbage characters” are ways most people I knew described what they saw there. But my curiosity to look further had the result that later, as a help desk assistant in the computer lab, I used the technique with one plain text editor or another to identify what program a file had been created with or to recover the essential text from a document that had become corrupted. Opening files with text editors was something that started as a curiosity but occasionally yielded other fruit… it became something I just do. By the time I first opened an Audacity *.aup file to find it contains not binary “gibberish” but well-formed XML I had received a training course in XML and had used XSLT to write a library catalogue management tool that was in use at the same lab, so I knew the discovery would come in handy.

When it was decided a 53 slide module I made was to be translated into 13 additional languages I found two very helpful applications for this bit of knowledge. The first, because I keep my projects in a MySQL database and identify every item on the storyboard by module, section, subsection, and slide, is that once the translations were loaded I was able to loop through a query and create a blank, properly tagged and labelled Audacity .aup file for every slide. The second is much more fun, so I’ll get the first one out of the way… I’m best with ColdFusion (CFML = ColdFusion Markup Language), but you can do this with PHP or any other language, just include or require the right libraries for parsing XML.

Create a new project in Audacity and save it as blank_project.aup. Open the file in any plain text editor (I like EditPadLite) What you do next will vary depending on what application server you’re using and how , but in ColdFusion my first go at it was pretty lazy. I put the entire file inside a cfsavecontent block, looped through a database query to get my information, plugged it in, and write it to the server’s hard drive. Of course that’s most practical if it’s also your laptop or desktop’s hard drive—I’ve done this with both Adobe’s wonderful (and free) ColdFusion developer version, and just recently with Railo, the open source CFML engine.(for PHP I’d use XAMPP, EZPHP or similar).

Your SQL will be your own, but it probably still helps to see mine…

SELECT * FROM allslides
 WHERE module = 'thisMod' AND lang = 'th-is'
 ORDER BY module_id, section_id, subsection_id, slide_id;

Loop through the query. n.b.: in XML it’s crucial to have no characters of any kind before the <?xml version="1.0" standalone="no" ?>When I said lazy, I meant it… to avoid trimming I just put it right after the opening cfsavecontent tag without a space or hard return (but if my source formatter tries to make it prettier I have to put it back). The variable “descript,” by the way, is a short human readable description I keep in the database so I don’t have to remember what slide hr101-B30 is, it comes out hr101-B30-keyconcepts. #SPEECH# is the word for “speech” in whatever language.

<cfloop query="myQuery"><!--- create a unique name for project,
                              file and data directory --->
<cfset thisProjectId = "#module_id#-#section_id##subsection_id##slide_id#-#descript#">
<!--- store the file in a variable --->
 <cfsavecontent variable="aup_file_contents"><?xml version="1.0" standalone="no" ?>
<!DOCTYPE project PUBLIC "-//audacityproject-1.3.0//DTD//EN"
 "http://audacity.sourceforge.net/xml/audacityproject-1.3.0.dtd" >
<project xmlns="http://audacity.sourceforge.net/xml/" projname="#thisProjectId#_data"
 version="1.3.0" audacityversion="2.0.1" sel0="0.0000000000" sel1="0.0000000000"
 vpos="0" h="0.0000000000" zoom="86.1328125000" rate="44100.0">
    <tags><!--- Replace #variables# with data from database --->
      <tag name="GENRE" value="#SPEECH#"/>
      <tag name="ARTIST" value="#MY_ORG#"/>
      <tag name="TRACKNUMBER" value="#currentRow#"/>
      <tag name="TITLE" value="#SECTION_TITLE# #SUBSECTION_TITLE# #SLIDE_TITLE# "/>
      <tag name="YEAR" value="#year(now())#"/>
      <tag name="ALBUM" value="#MODULE_TITLE#"/>
      <tag name="COMMENTS" value="(C) #year(now())#"/>
    </tags>
</project>
</cfsavecontent>
<!--- to save space, try/catch error-handling not shown --->
<cffile action="write" file="#DRIVE:\path\to\#thisProjectId#.aup"
        output="#aup_file_contents#">
<cfdirectory action="create" directory="DRIVE:\path\to\#thisProjectId#_data">
</cfloop>

That writes 53 files and creates 53 directories in about 3 seconds, well under a minute for all 13 languages if I place this loop in a loop of the languages. It saves me a lot of time.

My jSyncWithMedia plugin was a learning exercise for me, and I learned a lot. With jQuery it was easy for me to attach an “event-listener” to an HTML5 audio or video element’s timeupdate event, and create callbacks to change classes or run simple css animations depending on currentTime, even to enhance the display to show 1/10ths of seconds and get my timings more precise. But I knew all along no rational human would go through the tedious job of syncing all those items. It needs an interface, and while ideally that would be in JavaScript and built into the plugin—and perhaps one day it will be—the knowledge that an .aup is really just another a .xml brings out the laziness in me. You see, Audacity already has the exact interface I need… it’s the Labels track. I figure why not just borrow it?

The image shows the narration for a guitar lesson I’m doing in my own jSyncWithMedia-Alpha-1.0 plugin just because I want to. On the track labels I supply key:value pairs, where the key is always a valid html element (the “off:” key is ignored, but serves as a visual aid).

Showing Audacity's Labels track, spanning sections of audio, with values filled in

Audacity’s Labels track, spanning sections of audio, with values filled in that declare an element and its content.

When you add a label track and labels in Audacity you add <labeltrack> and <label></label> elements to the XML structure, with attributes for title, start time t and end time t1.

    <labeltrack name="Label Track" numlabels="11" height="253" minimized="0">
        <label t="1.69726544" t1="11.83946136" title="li:woodshed"/>
        <label t="11.92225480" t1="27.32183391" title="li:busting"/>
        <label t="20.79346939" t1="62.46764754" title="li:turntable"/>
        <label t="34.31787926" t1="43.54934739" title="li:changes the pitch"/>
        <label t="50.00723540" t1="62.46764754" title="a:audacity.soundforge.net"/>
        <label t="53.85713018" t1="62.50904425" title="img:audacity_logo.png"/>
        <label t="62.46764754" t1="62.55044097" title="off:ALL"/>
        <label t="76.70811855" t1="91.03138299" title="img:copy-paste.png"/>
        <label t="91.11417643" t1="109.90828642" title="li:copy menu item"/>
        <label t="110.15666673" t1="114.79309915" title="li:easily practice"/>
        <label t="114.79309915" t1="123.81758369"
               title="li:Choose File-&gt;Save Project As..."/>
    </labeltrack>

That’s all I need to create all the code to put my syncItems in a <div id=”jSWM”></div> and run my plugin on it to create a presentation.

Here’s the CFML

<cffile action="read" file="#request.pathToXml#\bust_a_solo.aup" variable="myFile">
<cfscript>
    // a debug tool
    dumpIt = request.dumpIt; // default request.dumpIt;

    // validate audacity
    scndLine = listGetAt(myFile,2,chr(10));
    isAudacityFile = REFind('^<!DOCTYPE project PUBLIC "-//audacityproject', scndLine );

    // Initialize strings to hold list and image content.
    contentSyncItemsList = '';
    contentImgDiv = '';

    // a function to display html on the page
    public string function printHTML( html ){
        var theHtml = arguments.html;
        var theHtmlFormatted = replaceList(theHtml,'<,>','&lt;,&gt;');
        return theHtmlFormatted;
    }

    // a function because I'm anal about pluralization'
    public string function pluralizeOn(required numeric n, boolean endsInY="false" ){
        var number = arguments.n;
        var y = arguments.endsInY;
        var plural = '';
        if(not number is 1) {
            plural = IIf( y, DE('ies'), DE('s') );
        } else  {
            plural = IIf( y, DE('y'), DE('') );
        }
        return plural;
    }
</cfscript>
<cfif isAudacityFile>
<cfscript>
 myXML = xmlParse(myFile);
 myLabels = xmlSearch(myXML, "//*[name()='label']");  
 myNamespace = "jswm"; // ohrc etc.
 n = arrayLen(myLabels);
 public string function getWord(required numeric index)  {
   return  myLabels[index].XmlAttributes['title'];
 };
 public string function getOnTime(required numeric index)  {
   return  numberFormat(myLabels[index].XmlAttributes['t'],'9.9');
 };
 public string function getOffTime(required numeric index)  {
   return  numberFormat(myLabels[index].XmlAttributes['t1'],'9.9');
 };
 elements = structNew();
</cfscript>
<cfoutput>
<h3>There are #n# labels in the Audacity file     .</h3>
<cfloop from="1" to="#n#" index="i">
<cfscript> // get the key:value pair that describes the item, then split into key and value
    thisItem = getWord(i);
    thisItemType = listFirst(thisItem,":");
    thisItemValue = listLast(thisItem,":");
    elements.elementType[i] = thisItemType; // FTR, I'm not yet using these 4 structures
    elements.elementContent[i] = thisItemValue; //  
    elements.elementOn[i] = getOnTime(i); //
    elements.elementOff[i] = getOffTime(i); //
    switch (thisItemType){ // format this into the HTML used by jSyncWithMedia
         case "a" : // NOTE: I BROKE VARIABLES ACROSS LINES to fit
          elements[i] = '<li data-#myNamespace#-on="#getOnTime(i)#"
           data-#myNamespace#-off="#getOffTime(i)#"><#thisItemType# 
           href="#thisItemValue#">#thisItemValue#</#thisItemType#></li>';
         break;
         case "img" :  // NOTE: I BROKE VARIABLES ACROSS LINES to fit
          elements[i] = '<#thisItemType# data-#myNamespace#-on="#getOnTime(i)#" 
          data-#myNamespace#-off="#getOffTime(i)#" 
          src="#request.pathToImgFolder##thisItemValue#" />';
         break;
         case "off" :
          elements[i] = 'offList="#thisItemValue#"';
         break;
         default: // li  // NOTE: I BROKE VARIABLES ACROSS LINES to fit
          elements[i] = '<#thisItemType# data-#myNamespace#-on="#getOnTime(i)#" 
          data-#myNamespace#-off="#getOffTime(i)#">#thisItemValue#</#thisItemType#>';
         break;
    }

</cfscript>
<cfif not thisItemType is "off">
<p><code>#thisItemType#</code> element "#thisItemValue#" shown at #getOnTime(i)#, 
           off at #getOffTime(i)# <br>
   </p><cfelseif thisItemType is "off"><!--- OFF post-processed here
                         Pass 'ALL' or comma-list of ordinal element indeces    --->
<cfif thisItemValue is 'ALL'><!--- loop through everything  
           // NOTE: I BROKE VARIABLES ACROSS LINES to fit  --->
<cfloop from="1" to="#i#" index="x">
    <cfset elements[x] = replace(elements[x],'data-#myNamespace#-off="99"',
              'data-#myNamespace#-off="#elements.elementOn[i]#"')></cfloop>
    <cfelse><cfset myArray = listToArray(thisItemValue)><!--- TODO: if keeping 
           deprecated off type must change to regexp --->
    <cfloop from="1" to ="#ArrayLen(myArray)#" index="i"><cfset thisElementNo = 
 myArray[i]><cfset elements[thisElementNo] = replace(elements[thisElementNo],
'data-#myNamespace#-off="99"',
'data-#myNamespace#-off="#getOnTime((structcount(elements) - 4))#"')></cfloop>
</cfif>
</cfif>
</cfloop>
</cfoutput>

<cfscript>
    elCount = (structcount(elements) - 4); //  <!--- Remove number of extra keys --->
    liCount = arrayLen(structFindValue(elements,'li','ALL'));
    imgCount = arrayLen(structFindValue(elements,'img','ALL'));
    aCount = arrayLen(structFindValue(elements,'a','ALL'));
</cfscript>
<cftry>
<cfsavecontent variable="contentSyncItemsList">
<cfoutput><!-- #liCount# li element#pluralizeOn(liCount)#, 
               containing #aCount# link#pluralizeOn(aCount)# -->
<ul id="syncItems">
<cfloop from="1" to="#elCount#" index="i"><cfif elements.elementType[i] is "li">
       &nbsp; &nbsp; #elements[i]#</cfif>
    </cfloop>
</ul></cfoutput>
</cfsavecontent>

<cfsavecontent variable="contentImgDiv">
<cfoutput><!-- #imgCount# img element#pluralizeOn(imgCount)# -->
<div id="syncImages">
<cfloop from="1" to="#elCount#" index="i">
    <cfif elements.elementType[i] is "img">&nbsp; &nbsp; #elements[i]#</cfif>
</cfloop>
</div></cfoutput>
</cfsavecontent>
<cfcatch>
    <cfif structKeyexists(cfcatch,"Detail")><cfset d=cfcatch.Detail>
      <cfelse><cfset d=""></cfif><cfoutput>
    <div><h3>#replace(cfcatch.Type,left(cfcatch.Type,1),ucase(left(cfcatch.Type,1)))#
              type error.</h3>
    <h4>#cfcatch.Message# #d#</h4>
    <p>Sometimes an error here means the variable 
       <code><strong>elCount</strong></code> is wrong. Did you change the number of 
        <code>elements["elementName"][i]</code> keys and not alter 
<code><strong>line 79</strong></code>?</p></div>
</cfoutput></cfcatch>
</cftry>

<!--- and here's the fruit of all that hard work --->
<cfoutput>
<div>
#printHtml(contentSyncItemsList)#

#printHtml(contentImgDiv)#
</div>
</cfoutput>

<cfif dumpIt><cfset findLIValues = structFindValue(elements.elementType,'li')>
<cfoutput>
    <p>#liCount#</p>
</cfoutput>
<cfdump var="#findLIValues#" expand="no" label="findLIValues">
    <cfdump var="#elements#" expand="no" label="ELEMENTS">    
</cfif>

<!--- DEBUGGING stuff... drag it up into the dumpIt area as needed
<cfdump var="#elements#" expand="no" label="elements">
<cfdump var="#getLabels[2]#" expand="no" label="getLabels[2]">
 <cfset myLabels = XMLSearch(myXML, "//*[name()='label']")>   
<cfoutput>#myLabels[1].XmlAttributes['t']#</cfoutput>
<cfdump var="#labels#" label="labels">  --->

<cfelse>
    <h3>Fatal error</h3>
    <h4>This does not appear to be a valid Audacity file.</h4>
</cfif>

If I add jQuery and my plugin to the page and place the CFML in a div, call jSWM() on it, I’ve got a presentation ready to go, and I can tweak the timings or add and remove events in Audacity. I only need the ColdFusion or Railo server on my own computer, copy/paste the generated code anywhere.

Image of output, jSWM generated dynamically from Audacity XML

jSWM generated dynamically from Audacity XML

For me this approach has the added benefit of forcing me to visualize my scenes more thoroughly and in advance, to organize my storyboards according to my design and vision (not just make things up as I go along) and then to stay true to the design and vision.

Aug 15

The current “state” of WAI-ARIA adoption and its “role” in accessibility

March 2014 UPDATE 2014: WAI-ARIA 1.0 is a completed W3C Recommendation as of March, 2014. Check current status

The WAI-ARIA project began in the fall of 2006. ARIA stands for “Accessible Rich Internet Applications,” by which it means web-based applications using CSS and JavaScript to change page elements without reloading the page. ARIA provides a means for the changes to communicate with assistive technologies.

“…I worked with a web project manager who was unfamiliar with ARIA, …and ended up interviewing half a dozen upcoming young developers, none of whom had heard of it either! Had the Web Accessibility Initiative’s initiative failed, …was ARIA D.O.A.?”

It came to my attention at that time due to my involvement with a group of teacher educators at the Faculty of Education at York U, Toronto. I admit I wasn’t able to make a great deal of sense of it until they published a Primer and a guide on Authoring Practice in 2010, and even so it remains daunting. Yet I believe in ARIA and what it’s trying to do, and I know of no other meaningful solution in the works. So I was disappointed and somewhat baffled when at my job in 2011 I worked with a web project manager who was unfamiliar with ARIA, and then, in the course of the project, ended up interviewing half a dozen upcoming young developers, none of whom had heard of it either! Had the Web Accessibility Initiative’s initiative failed, …was ARIA DOA?

Jutta Treviranus is Director and Professor at the Inclusive Design Institute at OCAD University. She’s explained at length the many challenges faced by people with differing abilities even if they’re using assistive technology, which involve availability, cost and compatibility issues far more convoluted than many of us may imagine. I recently had the chance to ask her some questions about ARIA adoption, and she’s graciously allowed me  to share her answers (and they let my colleagues off the hook!). Continue reading

Jan 10

ARIA adoption

Updated Problem mentioned her was fixed Nov 2013. …but I still can’t do the Show/Hide example here because the WordPress editor strips important attributes from <button> and <a> elements when you toggle from Visual to HTML mode. I got my Help rollovers working, though!Updated: see this post.

I don’t remember the details, but I stumbled upon the WAI-ARIA? project when it was a hatchling, and almost no one I knew had heard of it. It’s been over 2 years, the bird has left the nest, but it still seems to be flying under many people’s radar. It really should be an important piece in the designer’s repertoire. I’ve written about this at much greater length while I was doing an independent reading in Web Usability and Accessibility as part of my graduate studies. AJAX? messes with a web page’s “DOM”? in ways that are hidden from the user of the page by design. ARIA is meant to talk to assistive technologies and keep track of all such changes, communicating them to the software, or responding to custom designed hardware to do things those of us with so-called normal abilities take for granted. To do this, ARIA defines and then makes use of roles—e.g., this is a toolbar, this is a menu item; states—this panel is folded or invisible, this one is displayed; and properties—this menu has a drop-down, this one a pop-up.

Continue reading