Preventing Exchange from messing up multipart/alternative messages

[Mikhail Teterin]

Hello!

My cron-job generates nice two-part MIME messages (Content-Type:
multipart/alternative). The first part is text/plain with a short summary
(intended for people to read on their Blackberries).

The second part is text/html with details.

Unfortunately, something -- I suspect the Exchange server -- totally messes
up the message before they arrive to mailboxes:

1. The original text/plain part gets thrown-out completely,
no trace of it remains.
2. The text/html part gets re-processed -- it is recoded from the original
"us-ascii"/7bit into "iso-8859-1"/quoted-printable, and the text-only
part of its gets into a new text/plain part (which is utterly useless
for us).

If I'm right suspecting, this is all Exchange's fault, perhaps there is a
way -- some kind of header, maybe, that will tell this piece of junk to
STOP MESSING WITH THE BODIES of the arriving e-mails?

Thanks for any advice. Yours,

-mi

[Dmitry Streblechenko]

This is just how Exchange works, and I do not see anything wrong with this -
by definition, the receiving software can chose any of the
multipart/alternative parts.
Exchange always picks up the HTML part if it is available as it contains the
most information
MAPI in general and Exchange in particular do not allow the plain text body
(PR_BODY) to be different from the rich text (HTML or RTF - PR_HTML and
PR_RTF_COMPRESSED); they are always kept in sync.
If you want a short version of the message body, you can always request
PR_ABSTRACT (the first 255 charcters of the body) instead of PR_BODY.

Dmitry Streblechenko (MVP)
http://www.dimastr.com/
OutlookSpy - Outlook, CDO
and MAPI Developer Tool

Post by Mikhail Teterin
Hello!
multipart/alternative). The first part is text/plain with a short summary
(intended for people to read on their Blackberries).
The second part is text/html with details.
Unfortunately, something -- I suspect the Exchange server -- totally messes
1. The original text/plain part gets thrown-out completely,
no trace of it remains.
2. The text/html part gets re-processed -- it is recoded from the original
"us-ascii"/7bit into "iso-8859-1"/quoted-printable, and the text-only
part of its gets into a new text/plain part (which is utterly useless
for us).
If I'm right suspecting, this is all Exchange's fault, perhaps there is a
way -- some kind of header, maybe, that will tell this piece of junk to
STOP MESSING WITH THE BODIES of the arriving e-mails?
Thanks for any advice. Yours,
-mi

[Mikhail Teterin]

Post by Dmitry Streblechenko
This is just how Exchange works, and I do not see anything wrong with this
- by definition, the receiving software can chose any of the
multipart/alternative parts.

No, that's incorrect. Not the "receiving software", but the user agent:

 http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html

Exchange is NOT the user agent -- outlook, blackberry (software), kmail,
pine, thunderbird, &c. are the user agents. Exchange is the transport agent
and should NEVER modify the content of the messages.

Post by Dmitry Streblechenko
Exchange always picks up the HTML part if it is available as it contains
the most information

That is not up to the transport to decide. It is up to the composing
software.

Post by Dmitry Streblechenko
MAPI in general and Exchange in particular do not allow the plain text
body (PR_BODY) to be different from the rich text (HTML or RTF - PR_HTML
and PR_RTF_COMPRESSED); they are always kept in sync.

Hardly the first time a Microsoft product is found to be breaking
standards...

So I ask again, is there some way (perhaps through some header like
X-MS-Exchange-Keep-Off: true) to cause Exchange to let the message reach
each addressee's user agent unmodified?

-mi

[Dmitry Streblechenko]

That would be true if Exchange were a dumb storage database that simply
stored the MIME content of the messages.
Up to version 2000, it only stored messages broken into MAPI properties, the
original MIME source was discarded (if you don't count message headers).
Exchange 2000 and up can store either MIME or MAPI (or both) and convert
from one to another as necessary.
I don't know if it preserves the mismatched plain text and HTML parts in its
MIME store, but this is irrelevant in your case as Blackberry uses MAPI to
access the data, and in MAPI all flavors of the messages body are always
synchronized.

P.S. Please keep this discussion on the technical level: phrases like "piece
of junk" and "Hardly the first time a Microsoft product is found to be
breaking standards" say more you than any MS products.

Dmitry Streblechenko (MVP)
http://www.dimastr.com/
OutlookSpy - Outlook, CDO
and MAPI Developer Tool

Post by Mikhail Teterin

Post by Dmitry Streblechenko
This is just how Exchange works, and I do not see anything wrong with this
- by definition, the receiving software can chose any of the
multipart/alternative parts.

http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
Exchange is NOT the user agent -- outlook, blackberry (software), kmail,
pine, thunderbird, &c. are the user agents. Exchange is the transport agent
and should NEVER modify the content of the messages.

Post by Dmitry Streblechenko
Exchange always picks up the HTML part if it is available as it contains
the most information

That is not up to the transport to decide. It is up to the composing
software.

Post by Dmitry Streblechenko
MAPI in general and Exchange in particular do not allow the plain text
body (PR_BODY) to be different from the rich text (HTML or RTF - PR_HTML
and PR_RTF_COMPRESSED); they are always kept in sync.

Hardly the first time a Microsoft product is found to be breaking
standards...
So I ask again, is there some way (perhaps through some header like
X-MS-Exchange-Keep-Off: true) to cause Exchange to let the message reach
each addressee's user agent unmodified?
-mi

[Mikhail Teterin]

Post by Dmitry Streblechenko
Up to version 2000, it only stored messages broken into MAPI properties,
the original MIME source was discarded (if you don't count message
headers). Exchange 2000 and up can store either MIME or MAPI (or both) and
convert from one to another as necessary.

Are you confirming my fear, that there is no way to make Exchange deliver
the message without modifications?

Post by Dmitry Streblechenko
P.S. Please keep this discussion on the technical level: phrases like
"piece of junk" and "Hardly the first time a Microsoft product is found to
be breaking standards" say more you than any MS products.

Sir, my dislike for Microsoft products is, indeed, obvious, but they do, in
fact have major flaws -- as this very thread demonstrates: an e-mail sent
through Exchange is mangled en-route beyond recognition. A mail-transport
agent (MTA, which is what Exchange is acting as in this case), is not
supposed to do that*. Violating standards in this manner so would qualify
any piece of software as "piece of junk", however sophisticated it may be
in other respects.

Please, refrain from denying the obvious and from blaming me for pointing it
out. We have already established, that Exchange is acting contrary to the
standards -- the only remaining question is, whether anything can be done
to work around this problem in this particular case.

Although you did not see a problem here originally, you now know, there is a
problem. You saw my link to the RFC and accepted my correcting you, that,
indeed, Exchange should not be doing what it is doing. So why the personal
attack?

* There is a good reason for this standard -- any mangling should be
controlled by the recipient, but the recipients only have control over
their Mail User Agents (Outlook, Eudora, Thunderbird, Evolution, elm, pine,
Kmail, &c.). The Mail Transport Agents (such as Exchange, or sendmail, or
qmail, &c.) are outside of the users' control and thus should never modify
the body of the message.

-mi

[Dmitry Streblechenko]

Exchange is not sendmail etc. It does not function as a storage of the
(unmodified) MIME messages. I suggest that you at least have a cursory look
at one of the Exchange books such as
http://www.amazon.com/Microsoft-Exchange-Server-2007-Implementation/dp/1555583474/ref=sr_1_1/104-1513743-9996722?ie=UTF8&s=books&qid=1187147208&sr=8-1

When Outlook accesses a message in an Exchange folder, it does *not* (a-la
Eudora etc) request the whole MIME message or one of the MIME parts (if
FETCH in IMAP4 is used). MIME is not a native Outlook (or MAPI to be
precise) format. It does not speak MIME natively.
Whether you like that or not is irrelevant, this is just how things work.

To answer your question: since Blackberry uses MAPI (to the best of my
knowledge), you cannot force it to extract the original text/plain part.
My guess is that if it (Blackberry) were using POP3 or IMAP4, Exchange would
use its MIME storage and the unmodified message would be returned (unless it
was previously modified by a MAPI client, such as Outlook).

Nothing personal, but my friendly suggestion still stands: do educate
yourself first before using words like "piece of junk".
And if yor opinion is still the same, telling people who spend all their
time working with Exchange that it is a piece of junk is not the best way to
make friends (if you give a damn) or get an informative answer (if that is
all you really want).

Dmitry Streblechenko (MVP)
http://www.dimastr.com/
OutlookSpy - Outlook, CDO
and MAPI Developer Tool

Post by Mikhail Teterin

Post by Dmitry Streblechenko
Up to version 2000, it only stored messages broken into MAPI properties,
the original MIME source was discarded (if you don't count message
headers). Exchange 2000 and up can store either MIME or MAPI (or both) and
convert from one to another as necessary.

Are you confirming my fear, that there is no way to make Exchange deliver
the message without modifications?

Post by Dmitry Streblechenko
P.S. Please keep this discussion on the technical level: phrases like
"piece of junk" and "Hardly the first time a Microsoft product is found to
be breaking standards" say more you than any MS products.

Sir, my dislike for Microsoft products is, indeed, obvious, but they do, in
fact have major flaws -- as this very thread demonstrates: an e-mail sent
through Exchange is mangled en-route beyond recognition. A mail-transport
agent (MTA, which is what Exchange is acting as in this case), is not
supposed to do that*. Violating standards in this manner so would qualify
any piece of software as "piece of junk", however sophisticated it may be
in other respects.
Please, refrain from denying the obvious and from blaming me for pointing it
out. We have already established, that Exchange is acting contrary to the
standards -- the only remaining question is, whether anything can be done
to work around this problem in this particular case.
Although you did not see a problem here originally, you now know, there is a
problem. You saw my link to the RFC and accepted my correcting you, that,
indeed, Exchange should not be doing what it is doing. So why the personal
attack?
* There is a good reason for this standard -- any mangling should be
controlled by the recipient, but the recipients only have control over
their Mail User Agents (Outlook, Eudora, Thunderbird, Evolution, elm, pine,
Kmail, &c.). The Mail Transport Agents (such as Exchange, or sendmail, or
qmail, &c.) are outside of the users' control and thus should never modify
the body of the message.
-mi

[Mikhail Teterin]

Post by Dmitry Streblechenko
Exchange is not sendmail etc. It does not function as a storage of the
(unmodified) MIME messages.

Neither sendmail nor any of the other Mail Transport Agents "function as a
storage of the unmodified MIME messages". The only time a transport agent
would _store_ a message is in queue, while en-route to the destination. To
confuse you further, there are also Mail Delivery Agents (and sendmail is
not one of them either).

Exchange is combining the functions of the MTA and MDA, which is fine.

However, any message-mangling is the domain of a Mail User Agent (MUA), also
known as "mail client". Exchange doing so -- without even allowing for any
work-around -- is what qualifies it as "piece of junk".

I came to these forums with a question: "Is there a way to prevent Exchange
from mangling a multipart/alternative e-mail?" Instead of reading an answer
(such as: "Here is how ...", or: "No, you can not"), I am wasting time
educating you, what "multipart/alternative" is supposed to mean (no the
parts can not be regenerated en-route), and how an MTA is different from
Eudora. :(

Post by Dmitry Streblechenko
When Outlook accesses a message in an Exchange folder, it does not (a-la
Eudora etc) request the whole MIME message or one of the MIME parts (if
FETCH in IMAP4 is used).

Exchange's internal workings are not really relevant. If it speaks SMTP, it
should not mangle the messages. Imagine a post-office automatically opening
up and re-typing EVERY letter, and discarding the original (besides the
privacy concerns)... Most people would not notice (most only exchange typed
correspondence anyway). For the others this is outrage -- if only because
this causes a serious waste of time, electricity, and paper (or CPU-time
and storage space).

Post by Dmitry Streblechenko
MIME is not a native Outlook (or MAPI to be precise) format. It does not
speak MIME natively.

The above is not parsable. MIME is not a protocol to be "spoken".

Post by Dmitry Streblechenko
Whether you like that or not is irrelevant, this is just how things work.

... or don't work.

Post by Dmitry Streblechenko
To answer your question: since Blackberry uses MAPI (to the best of my
knowledge), you cannot force it to extract the original text/plain part.

Blackberry can use different protocols. Talking to Exchange it, likely, uses
MAPI, indeed. Talking to my own mail-server it uses IMAP4, though, and can
also speak POP3.

Post by Dmitry Streblechenko
My guess is that if it (Blackberry) were using POP3 or IMAP4, Exchange
would use its MIME storage and the unmodified message would be returned
(unless it was previously modified by a MAPI client, such as Outlook).

I'm afraid, your guess is wrong. Before I even reach for my Blackberry, I
can already see the message mangled message in my IMAP client...

Maybe, it is the Blackberry server connecting (via MAPI), that causes
Exchange to completely re-write the message... Interesting -- why would it
do that? And if it does, then MAPI itself is "piece of junk" (surprise!) --
retrieving a copy of a message should not be modifying its body on the
server.

Post by Dmitry Streblechenko
And if yor opinion is still the same, telling people who spend all their
time working with Exchange that it is a piece of junk is not the best way
to make friends (if you give a damn) or get an informative answer (if that
is all you really want).

My first post in this thread used the term "piece of junk" conditionally:
"If I'm right suspecting, this is all Exchange's fault ...". Now that you
-- an educated person, presumably -- have confirmed the suspicion (took you
several posts), I have all the grounds for my opinion.

Your continuing defense of this poorly-designed software lowers my opinion
about the people "who spend all their time" working with the piece of junk.
I can understand, how someone would do that for money (being an MVP must
have some tangible benefits), but to _sincerely_ defend it -- as you seem
to be doing -- is a bad sign...

Have a good one... As I posted separately, I found a work-around (no
solution seems to exist, unfortunately), and am unlikely to infringe upon
your peace again any time soon.

-mi

[Dmitry Streblechenko]

Does using Unix all day long have this effect on people? Jeeez.... Go easy
on coffee, would ya?

Dmitry Streblechenko (MVP)
http://www.dimastr.com/
OutlookSpy - Outlook, CDO
and MAPI Developer Tool

Post by Mikhail Teterin

Post by Dmitry Streblechenko
Exchange is not sendmail etc. It does not function as a storage of the
(unmodified) MIME messages.

Neither sendmail nor any of the other Mail Transport Agents "function as a
storage of the unmodified MIME messages". The only time a transport agent
would _store_ a message is in queue, while en-route to the destination. To
confuse you further, there are also Mail Delivery Agents (and sendmail is
not one of them either).
Exchange is combining the functions of the MTA and MDA, which is fine.
However, any message-mangling is the domain of a Mail User Agent (MUA), also
known as "mail client". Exchange doing so -- without even allowing for any
work-around -- is what qualifies it as "piece of junk".
I came to these forums with a question: "Is there a way to prevent Exchange
from mangling a multipart/alternative e-mail?" Instead of reading an answer
(such as: "Here is how ...", or: "No, you can not"), I am wasting time
educating you, what "multipart/alternative" is supposed to mean (no the
parts can not be regenerated en-route), and how an MTA is different from
Eudora. :(

Post by Dmitry Streblechenko
When Outlook accesses a message in an Exchange folder, it does not (a-la
Eudora etc) request the whole MIME message or one of the MIME parts (if
FETCH in IMAP4 is used).

Exchange's internal workings are not really relevant. If it speaks SMTP, it
should not mangle the messages. Imagine a post-office automatically opening
up and re-typing EVERY letter, and discarding the original (besides the
privacy concerns)... Most people would not notice (most only exchange typed
correspondence anyway). For the others this is outrage -- if only because
this causes a serious waste of time, electricity, and paper (or CPU-time
and storage space).

Post by Dmitry Streblechenko
MIME is not a native Outlook (or MAPI to be precise) format. It does not
speak MIME natively.

The above is not parsable. MIME is not a protocol to be "spoken".

Post by Dmitry Streblechenko
Whether you like that or not is irrelevant, this is just how things work.

... or don't work.

Post by Dmitry Streblechenko
To answer your question: since Blackberry uses MAPI (to the best of my
knowledge), you cannot force it to extract the original text/plain part.

Blackberry can use different protocols. Talking to Exchange it, likely, uses
MAPI, indeed. Talking to my own mail-server it uses IMAP4, though, and can
also speak POP3.

Post by Dmitry Streblechenko
My guess is that if it (Blackberry) were using POP3 or IMAP4, Exchange
would use its MIME storage and the unmodified message would be returned
(unless it was previously modified by a MAPI client, such as Outlook).

I'm afraid, your guess is wrong. Before I even reach for my Blackberry, I
can already see the message mangled message in my IMAP client...
Maybe, it is the Blackberry server connecting (via MAPI), that causes
Exchange to completely re-write the message... Interesting -- why would it
do that? And if it does, then MAPI itself is "piece of junk"
(surprise!) --
retrieving a copy of a message should not be modifying its body on the
server.

Post by Dmitry Streblechenko
And if yor opinion is still the same, telling people who spend all their
time working with Exchange that it is a piece of junk is not the best way
to make friends (if you give a damn) or get an informative answer (if that
is all you really want).

"If I'm right suspecting, this is all Exchange's fault ...". Now that you
-- an educated person, presumably -- have confirmed the suspicion (took you
several posts), I have all the grounds for my opinion.
Your continuing defense of this poorly-designed software lowers my opinion
about the people "who spend all their time" working with the piece of junk.
I can understand, how someone would do that for money (being an MVP must
have some tangible benefits), but to _sincerely_ defend it -- as you seem
to be doing -- is a bad sign...
Have a good one... As I posted separately, I found a work-around (no
solution seems to exist, unfortunately), and am unlikely to infringe upon
your peace again any time soon.
-mi

[Mikhail Teterin]

Mikhail Teterin wrote:

[Answering myself, because nobody else would]

Post by Mikhail Teterin
If I'm right suspecting, this is all Exchange's fault, perhaps there is a
way -- some kind of header, maybe, that will tell this piece of junk to
STOP MESSING WITH THE BODIES of the arriving e-mails?

The way I found to work-around the problem is the following: use
multipart/mixed instead of multipart/alternative, so that the HTML content
is an attachement, rather than a possible alternative of the message body
(which is now text). Thus my generated message now consists of two parts: a
short text/plain with a total time, and a large text/html with the detailed
report.

Of course, the Exchange can't just leave the message alone -- the short
text/plain part of the original itself gets replaced with a multipart/mixed
subpart consisting of the original plain text and its HTML version. Also,
the HTML report (originally in 7bit) gets gratuitously re-generated into
quoted-printable -- I'm seeing all of these parts in my IMAP client
(KMail).

But, at least, no text is thrown out and both the text/plain and the
text/html parts of the original get delivered to the recipients. Hurrah!

-mi

[Oliver Moazzezi [MVP]]

Hi,

Post by Mikhail Teterin
2. The text/html part gets re-processed -- it is recoded from the original
"us-ascii"/7bit into "iso-8859-1"/quoted-printable, and the text-only
part of its gets into a new text/plain part (which is utterly useless
for us).

You can force Exchange to keep the message as the recieved 7bit MIME type,
read:

http://support.microsoft.com/kb/257569/


Possibly interesting links for you:

'Frequently asked questions about MIME and content conversion in Exchange
2000 Server and in Exchange Server 2003'

http://support.microsoft.com/kb/836555


'Understanding Content Conversion'

http://technet.microsoft.com/en-us/library/bb232174.aspx


I will agree Exchange Server doesn't always conform to RFC standards. I've
had similar issues with 7bit/8bit MIME conversion. However you really do
need to lay off the coffee, you do come across quite rude and arogant.

Oliver