Discussion:
Log shipping
(too old to reply)
WLAN
2008-06-24 11:32:01 UTC
Permalink
Hello All,

I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records in it.

I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.

I have the following doubts with log shipping:

(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET ,EXOLEDB etc

I am trying some reseach on this transaction logs. Any idea is welcome.

Thanks in advance

regards
~WLAN
Andy David {MVP}
2008-06-24 13:33:24 UTC
Permalink
On Tue, 24 Jun 2008 04:32:01 -0700, WLAN
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records in it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET ,EXOLEDB etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
What is the goal of this project?
Why the need to read logs?
WLAN
2008-06-25 03:15:00 UTC
Permalink
Hello,

I want to archive mails without making footprint of Exchange server. Is it
possible to read mails with 'zero" footprint of exchange server?

Regards
wlan
Post by Andy David {MVP}
On Tue, 24 Jun 2008 04:32:01 -0700, WLAN
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records in it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET ,EXOLEDB etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
What is the goal of this project?
Why the need to read logs?
John Fullbright
2008-06-25 14:57:55 UTC
Permalink
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records in it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET ,EXOLEDB etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
WLAN
2008-06-26 03:16:18 UTC
Permalink
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records in it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET ,EXOLEDB etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
John Fullbright
2008-06-26 20:52:48 UTC
Permalink
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to the
Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records
in
it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
Martin Blackstone
2008-06-26 21:16:44 UTC
Permalink
LOL.
After thinking about this for 24 hours, I think he means he doesn't want to
touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to the
Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records
in
it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
John Fullbright
2008-06-26 21:37:07 UTC
Permalink
Keeps saying "without making a connection to the exchange server". How did
they intend to get the logs?


Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to the
Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited records
in
it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
Martin Blackstone
2008-06-26 21:50:03 UTC
Permalink
I told you, I'm out of ideas. :)
Post by John Fullbright
Keeps saying "without making a connection to the exchange server". How
did they intend to get the logs?
Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to
the Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited
records in
it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
unknown
2008-06-27 00:55:39 UTC
Permalink
Just wait a little while guys - quantum tunneling is planned for the next
version of Exchange.
Post by Martin Blackstone
I told you, I'm out of ideas. :)
Post by John Fullbright
Keeps saying "without making a connection to the exchange server". How
did they intend to get the logs?
Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to
the Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited
records in
it.
I want to read mail messages without accessing/connecting exchange server.
So my idea is to read tranaction logs. But right now i don't know the file
format of that file. I think it is a JET file database / transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
WLAN
2008-06-27 03:56:01 UTC
Permalink
Hello,

I think u misunderstood it.

I know that transaction logs are propretery info and only server / backup
APIs might know about its format. I am searching that whether any 3rd party
can access those log files.

My goal is to retrieve a message from the transaction logs, and thus, avoid
making a client connection to Exchange to retrieve the message – to enable us
to support a “zero footprint” interface with Exchange.

Lot of e-mail archiving softwares uses this mechanism for archiving e-mails
into repository

I know that i can read all messages, if I establish a connection to exchange
server. Also Journaling creates message traffic and need a connection to the
database.
Based on the above feedbacks and assumptions, I am doing a feasiblity study
about mail store. I have see mechanisms like Volume Shadow Copy Service
(VSS), Backup API etc. VSS creates a shadow copy of the mail store and we can
restore that copy into a non - exchange repository also.
Post by unknown
Just wait a little while guys - quantum tunneling is planned for the next
version of Exchange.
Post by Martin Blackstone
I told you, I'm out of ideas. :)
Post by John Fullbright
Keeps saying "without making a connection to the exchange server". How
did they intend to get the logs?
Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to
the Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you touch them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of exchange server.
Exchange server creates transaction logs and keeps uncommited
records in
it.
I want to read mail messages without accessing/connecting exchange
server.
So my idea is to read tranaction logs. But right now i don't know
the file
format of that file. I think it is a JET file database /
transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
John Fullbright
2008-06-27 13:47:00 UTC
Permalink
1. Reading the log files directly would avoid a MAPI or other client
connection, however it is not zero footprint. You still connect to the
server and you still place an IO load on the server as you read the logs.

2. Backup API backups are not zero footprint. In fact accounting for the
IO caused by backups is a large part of sizing an exchange server.

3. Copy on Write snapshots, like those created by the MS Software provider,
are not zero footprint. While a COW snapshot is in place, each overwrite
requires 3 IOs (read old data, write it to the difference area, overwrite
changed data). Put more than a few in place and your server will come to a
screaching halt.

4. Most Archiving tools I've seen use an Event sink. This goes way back to
the Archiving Sink example in the 2000 reskit and later Exchange 2000 SP2.
http://support.microsoft.com/kb/307798/en-us

5. Even the Archiving sink is not zero footprint. It takes IO to read the
messages and send content somewhere, and that IO will impact your server.
Post by WLAN
Hello,
I think u misunderstood it.
I know that transaction logs are propretery info and only server / backup
APIs might know about its format. I am searching that whether any 3rd party
can access those log files.
My goal is to retrieve a message from the transaction logs, and thus, avoid
making a client connection to Exchange to retrieve the message - to enable
us
to support a "zero footprint" interface with Exchange.
Lot of e-mail archiving softwares uses this mechanism for archiving e-mails
into repository
I know that i can read all messages, if I establish a connection to exchange
server. Also Journaling creates message traffic and need a connection to the
database.
Based on the above feedbacks and assumptions, I am doing a feasiblity study
about mail store. I have see mechanisms like Volume Shadow Copy Service
(VSS), Backup API etc. VSS creates a shadow copy of the mail store and we can
restore that copy into a non - exchange repository also.
Post by unknown
Just wait a little while guys - quantum tunneling is planned for the next
version of Exchange.
Post by Martin Blackstone
I told you, I'm out of ideas. :)
Post by John Fullbright
Keeps saying "without making a connection to the exchange server".
How
did they intend to get the logs?
Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to
the Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you
touch
them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of
exchange
server.
Exchange server creates transaction logs and keeps uncommited
records in
it.
I want to read mail messages without accessing/connecting exchange
server.
So my idea is to read tranaction logs. But right now i don't know
the file
format of that file. I think it is a JET file database /
transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is welcome.
Thanks in advance
regards
~WLAN
WLAN
2008-06-30 08:27:00 UTC
Permalink
Hello,

Thanks for the reply.

I have seen your points. Can you tell me the mechanism for reading the log
files directly.

Now i don't know the format of transaction log file. Is there any mechnism
for reading those logs?

regards
WLAN
Post by John Fullbright
1. Reading the log files directly would avoid a MAPI or other client
connection, however it is not zero footprint. You still connect to the
server and you still place an IO load on the server as you read the logs.
2. Backup API backups are not zero footprint. In fact accounting for the
IO caused by backups is a large part of sizing an exchange server.
3. Copy on Write snapshots, like those created by the MS Software provider,
are not zero footprint. While a COW snapshot is in place, each overwrite
requires 3 IOs (read old data, write it to the difference area, overwrite
changed data). Put more than a few in place and your server will come to a
screaching halt.
4. Most Archiving tools I've seen use an Event sink. This goes way back to
the Archiving Sink example in the 2000 reskit and later Exchange 2000 SP2.
http://support.microsoft.com/kb/307798/en-us
5. Even the Archiving sink is not zero footprint. It takes IO to read the
messages and send content somewhere, and that IO will impact your server.
Post by WLAN
Hello,
I think u misunderstood it.
I know that transaction logs are propretery info and only server / backup
APIs might know about its format. I am searching that whether any 3rd party
can access those log files.
My goal is to retrieve a message from the transaction logs, and thus, avoid
making a client connection to Exchange to retrieve the message - to enable
us
to support a "zero footprint" interface with Exchange.
Lot of e-mail archiving softwares uses this mechanism for archiving e-mails
into repository
I know that i can read all messages, if I establish a connection to exchange
server. Also Journaling creates message traffic and need a connection to the
database.
Based on the above feedbacks and assumptions, I am doing a feasiblity study
about mail store. I have see mechanisms like Volume Shadow Copy Service
(VSS), Backup API etc. VSS creates a shadow copy of the mail store and we can
restore that copy into a non - exchange repository also.
Post by unknown
Just wait a little while guys - quantum tunneling is planned for the next
version of Exchange.
Post by Martin Blackstone
I told you, I'm out of ideas. :)
Post by John Fullbright
Keeps saying "without making a connection to the exchange server".
How
did they intend to get the logs?
Anyway, why wouldn't they just journal the mailboxes in question.
Post by Martin Blackstone
LOL.
After thinking about this for 24 hours, I think he means he doesn't want
to touch the store.
After that point I'm pretty much out of ideas.
Post by John Fullbright
The logs reside on the the Exchange server. Unless your application
exploits quantum tunneling, to read them, you will need to connect to
the Exchange Server.
Post by WLAN
I think you misunderstood it. I want to read mails without making a
connecttion to exchange server.
Post by John Fullbright
The transaction logs are stored on the exchange server. If you
touch
them,
you are connecting to/touching the exchange server.
Post by WLAN
Hello All,
I have seen the documentation of log shipping feature of
exchange
server.
Exchange server creates transaction logs and keeps uncommited
records in
it.
I want to read mail messages without accessing/connecting exchange
server.
So my idea is to read tranaction logs. But right now i don't know
the file
format of that file. I think it is a JET file database /
transaction file.
(1) Is there any mechanism for reading these transaction logs?
(2) Can we read log file using APIs like Backup/restore,ESE / JET
,EXOLEDB
etc
I am trying some reseach on this transaction logs. Any idea is
welcome.
Thanks in advance
regards
~WLAN
Oliver Moazzezi [MVP]
2008-07-01 13:23:21 UTC
Permalink
Post by WLAN
Thanks for the reply.
I have seen your points. Can you tell me the mechanism for reading the log
files directly.
Now i don't know the format of transaction log file. Is there any mechnism
for reading those logs?
Hi there,

You need to give up on the idea of reading the Transaction Logs. Implement
Journaling and then read any messages from an alternate Journaling Exchange
Server.

Oliver
Andy David {MVP}
2008-07-01 13:30:16 UTC
Permalink
On Tue, 1 Jul 2008 14:23:21 +0100, "Oliver Moazzezi [MVP]"
Post by Oliver Moazzezi [MVP]
Post by WLAN
Thanks for the reply.
I have seen your points. Can you tell me the mechanism for reading the log
files directly.
Now i don't know the format of transaction log file. Is there any mechnism
for reading those logs?
Hi there,
You need to give up on the idea of reading the Transaction Logs. Implement
Journaling and then read any messages from an alternate Journaling Exchange
Server.
Oliver
Not to mention that the logs arent "Joe Smith sent a message to Bob"
Its more like , "At offset 3bf543......"
John Fullbright
2008-07-01 20:32:41 UTC
Permalink
If you really wanted zero impact, then an archiving appliance that sits in
the middle is about the only thing I can think of. If the appliance were a
Windows box, you'll still implement an event sink of sorts that copied off
the RFC822 message and then manipulated it till your heart's content as it
transited the SMTP queue. For items locally delivered to the store still in
MAPI format (Exchange 2000/2003), you'd have to route them through the
appliance (border server) to force content conversion. #*&$ Still not zero
footprint, is it?
Post by Andy David {MVP}
On Tue, 1 Jul 2008 14:23:21 +0100, "Oliver Moazzezi [MVP]"
Post by Oliver Moazzezi [MVP]
Post by WLAN
Thanks for the reply.
I have seen your points. Can you tell me the mechanism for reading the log
files directly.
Now i don't know the format of transaction log file. Is there any mechnism
for reading those logs?
Hi there,
You need to give up on the idea of reading the Transaction Logs. Implement
Journaling and then read any messages from an alternate Journaling Exchange
Server.
Oliver
Not to mention that the logs arent "Joe Smith sent a message to Bob"
Its more like , "At offset 3bf543......"
Andy David {MVP}
2008-07-01 13:28:47 UTC
Permalink
Is there any mechnism for reading those logs?
Yes. Exchange.
Loading...