So… a bit of background: Outlook is a terrible program, and reasonable people might choose to use something better to interact with their emails. In so doing, one often has to fall back on the IMAP protocol to interact with one’s Exchange server.
And, if one’s Exchange server is hosted on Office 365, it appears that all of your data is served up via IMAP. By “all”, I mean not just email (which you expect), but also contacts and calendar events. In general, these should show up on the eponymous folders within your directory (“Contacts” and “Calendars”, and various subfolders within that). This means that your email program can, potentially, allow you to do things with your contacts and calendar events as though they were emails… things that Exchange/Office 365 might not necessarily recognize as “valid” and could result in serious data corruption and deletion. This. Is. Bad.
Suppose one connected a client such as MailSpring via IMAP to Office 365. Hypothetically. Not that anyone near and dear to me might have done such a thing, but let’s suppose. It turns out that MailSpring will (correctly) recognize that the calendar events and contacts are not mail messages, and then fail to download them. In fact, the various folders would be filled with messages (one per contact or calendar event) that read:
Retrieval using the IMAP4 protocol failed
This looks like junk. So one might be forgiven for, say, deleting the message. Curiously, MailSpring then deletes the original (unretrieved) file from the server… deleting the contact or calendar event.
As an aside: I observe that many other mail clients (Outlook, Mail.app, etc.) choose to simply hide these files, rather than presenting this confusing message.
As a further aside: these IMAP-error-message messages, in our scenario, confusingly appeared in the Sent Items folder, rather than a calendar folder. No explanation has thus far been found for this…
So. Maybe a few hundred or thousand calendar events get deleted from an Office 365 account. How to get them back?
In the end, it was a multi-step process:
This takes us through midweek. At this point, it appears that this process is not 100% successful: new edits to the calendar are not always “sticking”. Our surmise is that Outlook views the calendar files as corrupted. This is further complexified by the fact that not one, but several calendars were involved. The subscribed calendars come from a separate Outlook.com account (not Office 365) and changes (including the original deletions) seem not to have transfered consistently upstream (although some new additions and moves from the default calendar have).
This is a pain.
Our next step, we think, will be to export the default calendar to an ICS file, unshare the Outlook.com files with the Office 365 account, and basically purge all calendar data from the Office 365 account. Then we’ll reimport the ICS file of exported items to the calendar, and reshare the calendars from Outlook.com.
Confidence is not super-high. But I figured that posting this process could be helpful to others in our boat.
Seth Battis May 10th, 2019
Posted In: How To
I’m working at a school using Office 365, and OneNote is the main event. I notice, however, that there don’t seem to be many clear, declarative answers to questions that I have about OneNote Class Notebooks, so I did some fiddling to find out.
As of the date of this post, the following questions and answers are empirically true of OneNote 2016 and the most current version of the Class Notebook plugin:
No (mostly). The motivation for this was that it would be really keen for class notebooks to live in our departmental shared OneDrive folder, so that we can both archive old classes and peek into each other’s notebooks with ease. It turns out that class notebooks can only be created in your personal OneDrive, in the Class Notebooks directory.
If you create the class notebook and then move it to a more desireable location (via OneNote’s ability to sync a notebook to a different location), it will totally make a copy of your notebook in that new location, but all of the students will still be accessing the original location, so it’s not gonna work for actually moving the OneNote.
If you move the class notebook (i.e. sync it to a new location and delete the original), you can no longer manage the notebook via the web interface that you need to use to add/remove students, lock the Collaboration Space, generate parent/guardian links, etc., etc. Essentially, it is dead to the web management interface. (But this does seem like a good way of archiving old class notebooks to a shared space at the end of the year.)
If you move your class notebook back to the Class Notebooks directory where it is supposed to be, the web management interface will not recognize it as a class notebook. And there is no path for (re)converting a seemingly normal OneNote notebook into a class notebook (again).
So, basically: no.
Sort of, but it’s probably not a good idea.
I really don’t like “_Collaboration Space” and “_Content Library” (or “_Teacher Only”), which are the default names of those sections in class notebooks (and for which there is no management interface for renaming them).
You can, of course, just rename them in OneNote. And they’ll get re-alphabetized into the mix with the students’ section groups. And they seem to continue to behave with the proper permissions. Which is good. However, the Class Notebook plugin will now interpret those sections as being additional students (who seem to not have the right sections within their section group, to boot), so that when you try to distribute a page to your students, you need to be careful to not distribute it into your renamed Collaboration Space or Content Library. This is a little confusing.
If you rename them back to their original names, the Class Notebook plugin reverts to treating them as “special” section groups that are not representative of students.
Probably better not to rename them.
You can now create these nifty little collaboration sections within your Collaboration Space that are limited to a specific group of students. You have to create these sections through the web interface. By default, these new collaboration sections are created at the root of the Collaboration Space.
If you move these sections into a new section group (e.g. “Last Week’s Groups”), the permissions appear to carry over into their new location.
In fact, if there are section groups created inside the Collaboration Space, you will have the option of creating new collaboration sections directly inside those section groups, through the web interface. W00t!
These are created through the web interface.
It legit only gives access to that particular student’s section group. None of the shared section groups (Collaboration Space, Content Library) are visible through that link.
I did not explore what happens if you post a link into one of those section groups into a student section group… I doubt the link would be follow-able.
Also created through the web interface.
This is also tightly restricted: just the Content Library. There doesn’t seem to be a built-in mechanism for sharing access to the Collaboration Space.
Seth Battis August 31st, 2018
Posted In: How To
For the past few years (well, since I’ve had an iPad), I’ve been exasperated by the time it takes whichever app Apple has streaming video from my iTunes library to my iPad to load my iTunes library. In the past, it has taken anywhere from 20-90 minutes to load the library each time I access it. Clearly, this isn’t really viable as a “streaming TV in bed” approach. I did some digging around, and it seems like the root issue is that my iTunes library is too big, so the iPad app can’t process the XML file it has to download (aside: this is ridiculous, of course).
Long story short, at the start of the summer, being blissfully unemployed, I took a moment to split my iTunes library in twain: one library of music and audiobooks to sync to my phone (not that that integration has been getting any better, mind you) and one library of movies and TV shows to stream to the AppleTV and my iPad and whatnot. (Note: this whole time the AppleTV could load my gigundous library… but not the iPad. Oy.)
Much metadata cleaning later (because it turns out that iTunes doesn’t actually write all the fields it should write to video files, because, well… it’s lousy), I now have two libraries. Most of the time I want the movies and TV library open, but I want to regularly open the music and audiobooks to sync podcasts (not that I trust Apple for that either: greg is in charge of syncing, iTunes just adds the new files to the library).
In the past, I’ve noted that I’ve found CLI
defaults tool to be useful for managing a remote iTunes library. This proves not to be the case here — in fact, the solution is one of simple symlinking. I’ve written three scripts: a “master” script that just toggles between libraries and two “client” scripts that just pass a specific parameter to the master script (depending on which library should be opened).
August 12, 2017 edit: I have found that AppleScript is just too flaky for this operation, and have switched to an end run around iTunes using bash scripting. The original AppleScripts are still available, but I recommend the bash approach below at this point:
It’s worth noting a few things here:
~/Music/iTunes— the script operates by swapping out symlinks at that location to other libraries. The script does do a sanity check before trying to effect that swap, of course.
osascriptcommand to open the “master” script with appropriate parameters in the crontab… I couldn’t figure out how to give osascript appropriate permissions in the accessibility section of the Security & Privacy system preferences. Frustrating. [August 12, 2017 edit: …and I no longer both to schedule switching libraries with crontab anyway.]
Seth Battis July 30th, 2017
Posted In: How To
Apparently they were introduced in Apache 2.3, and they are wicked useful, it turns out. At least, wicked useful if what you want to do is figure out what the URL of a to a specific file on the server is, assuming that you have only its path and some other file’s URL.
What follows seems to be empirically correct, but I welcome corrections:
DOCUMENT_ROOT is the system file path to the root of the web server’s document directory (usually something like
But, imagine that per-user web directories have been enabled on your server. That is, if my user is foo, then there is a directory
/home/foo/public_html in which all of foo’s web documents are stored (and served from). This is (probably) outside of the main server document root directory. The URL of foo’s web files would be something like
And this is where our variables enter the story.
CONTEXT_PREFIX is the portion of the
REQUEST_URI that triggered the server to serve up a file outside of the document root. And
CONTEXT_DOCUMENT_ROOT is the root directory that that particular CONTEXT_PREFIX is rooted at. Nota bene: conveniently
CONTEXT_DOCUMENT_ROOT contains the path the document root for whichever context this particular script was accessed through (so it would be the same as
DOCUMENT_ROOT within the server’s document root, while
CONTEXT_PREFIX would be an empty string, since there is no context prefix within the server’s document root.
Consider this extended example:
User foo has placed files in that user’s public_html directory. There are nested directories within that public_html directory, and we are accessing a file at this URL:
This file is stored at the following path in the file system:
Thus, a sampling of relevant server variables (presented in
$_SERVER for the PHP users amongst us) would be:
Alternatively, a similar example within the server’s document root:
is a URL that access a file at:
This results in the following values:
(the empty string)
For giggles, I cooked up some code to generate the URL of particular directory based on this information. I used the following little snippet to do this in
DataUtilities::overlap() in battis/data-utilities:
Seth Battis March 30th, 2016
Posted In: How To
So, my students in Advanced CS are working on their second semester projects. One project group is working on mastering Unity to build an iPhone game, and they just spent several days struggling with getting Unity Remote to work with their iPhones.
After much Googling and exasperation, they figured out the answer (which was not, prior to this post.
They had been able to see the iPhone in the Unity editor. They had been able to set up the remote to use the iPhone in the Unity editor in project settings. But when they “played” their game in the editor, it just sat there inertly, waiting for a connection. Consternation. Rending of garments. So forth and so on.
The trick, it turns out, is that when one installs the Unity editor, one must install iOS development support. Not a shocker. But certainly worth mentioning in an error message somewhere.
Seth Battis March 11th, 2016
Posted In: How To
I just mentioned that my wife switches between two different iTunes libraries, including one that lives on a shared volume on our home server. There’s a great tip on Stack Overflow for writing a script to automate that swap between iTunes libraries (rather than having to hold down option while iTunes loads and then click around to find the library you want). But it seemed like it might be nice to document for posterity the full script that we’ve built, since it handles not just the library swapping, but also the mounting of the shared volume as well.
This script takes a little prep work, both generating the library location property and storing the authentication information in the user keychain. The Stack Overflow tip explains the former, and the latter is as easy as connecting to the server and remembering to check the “Save Authentication” checkbox during the authentication process.
Seth Battis March 18th, 2014
Posted In: How To
A bit of background: my wife has an iTunes library that is too large to fit comfortably on her MacBook Air (in all fairness: I have an iTunes library that is too large to fit on her MacBook Air, my MacBook Pro and several other laptops as well). She maintains a local library on her laptop for use at school (useful movies, songs, etc.) and stores her “real” library on our home server. Which involves a bit of fiddling to get it to work. The key issue is that she needs iTunes to load her library off of a shared disk. And, periodically (maybe every couple of months), something goes haywire and she gets this error:
Needless to say, this is exasperating.
There are a number of suggested solutions out there, and I keep forgetting to bookmark “the one” that works. On my last search, I realized that this was because none of them are really the answer that works in our situation. Here’s what works (and I’m posting this as a note for self):
sudo chflags -R nouchg
At least, that’s what worked this evening, and it felt really familiar. (There’s an unspoken step 2a, of course: cancel the multi-hour process on the server that you had queued up moments before iTunes started misbehaving. Argh.)
Seth Battis March 18th, 2014
Posted In: How To
So many of these entries are really just an attempt to make sure that the next time I go searching for an answer to some question, by gum there will be a search result. That is: I write up things that I couldn’t find an easy answer to, so that someone else reaps the benefit of my suffering.
But, sometimes, it’s just about admitting that I’m dumb.
Case in point: I’ve been grousing on Twitter about a number of things in the past week, one of which was that, for much of the fall, many of my Google calendars have been showing up in iCal doubled. Every event shows up twice. Sometimes more than that, when Google has a hiccup. And birthdays from my contacts have shown up as many as 34 times — no joke: I counted. But the core problem has been that I’ve been seeing many of my Google calendars twice in iCal. It makes me feel busy, but otherwise it serves no purpose.
And then it hit me this evening, as I was looking at the list of calendars in iCal: with the launch of Mountain Lion, Apple revised how it handled Mail, Contacts & Calendars — so much so, in fact, that they created a separate prefpane for it. That connects to iCal, Mail and the Address Book pretty transparently. Before that upgrade, to see secondary Google calendars (the ones other than your primary calendar — a distinction about which I have some more grousing to do at a later date), you had to enable calendar delegates in the Accounts section of the iCal preferences. It was messy, and ugly (you had both the delegate and the calendar nested in the delegate… for each and every calendar). But it worked.
I never turned off the delegates when Apple upgraded to Mail, Contacts & Calendars. So I was seeing both the delegate calendar and the calendar associated with my Google account in Mail, Contacts & Calendars.
I just turned off delegates and everything got better. Except maybe those duplicated birthdays… we’ll see what happens with them.
Seth Battis December 9th, 2012
Posted In: How To
I’m about to embark on a group video project in my New Media class. To that end, it’s a hassle when the video (and script!) exist only in the account of one of the group members. I saw one suggestion for how to store the iMovie events and projects on an external drive, which seemed deeply convoluted and, well… a hassle. I’m not interested in purchasing or setting up external drives, or in going through that rigamarole to set it up.
So, it occurred to me that /Users/Shared/ is a pretty fine place to store shared files. And that the modern Mac OS X seems to honor symlinks pretty regularly (something I use all the time to sync my life across machines using dropbox — symlinking whole directories to sync preferences, settings and documents that the developers haven’t — yet — moved to the cloud.)
It seems to work for iMovie as well. Here’s how I did it (stay tuned to see if it blows up on my students!):
chmod -R a+rw /Users/Shared/iMovie\ Events
ln -s /Users/Shared/iMovie\ Events ~/Movies/iMovie\ Events
(Note that the space in “iMovie Events” needs to be escaped with a backslash!)
In my limited testing, this seems to work transparently. The biggest caveat is the permissions change. When creating events, iMovie seems to strip the write permissions off of the inherited permissions for the iMovie Events folder (but other users can still read and execute, which should be fine). Similarly, it’s possible that other users can only read (but not edit) the shared project.
We shall see.
Seth Battis October 31st, 2012
Posted In: How To