added (optional) encryption to message metadata and local blob storage

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?

[johnwiesel]

I have created this new branch (to replace feature-encryptall) to create a clean git history to allow easier review and cleaner merging.

This branch aims to allow users to activate encryption of blob data, even if the blob is stored in cassandra. I also allows to store all parsed message details ("metadata") like body, addresses, etc. using the same encryption.

[johnwiesel]

Any news? I'll be travelling by the end of next week. If there are any questions I'll answer them as soon as I return.

[rstml]

There are few concerns I have:

1. Metadata encryption. It seems that each header field has to be encrypted separately as well as parsed text/html. I believe we could simplify this but we need to change original design slightly. When EI was initially designed we decided to store each header as a separate column. However, this is unnecessary as they always read/write together. We could have stored all headers in a single column with serialized value. For encryption scenario this would be better approach.
2. Message class shouldn't know anything about encryption. Encryption should be handled on the persistence layer transparently.

These are relatively large changes and it would be good to postpone them until 0.5 release. We'll probably push out 0.4 soon.

[johnwiesel]

Alright! I am glad to hear that these changes will be integrated, even it is not in the next release.

I was wishing to deploy an EI instance for production use pretty soon. But since point 1 will ensue a change in the data model, this might have to wait until 0.5.. Is there any timeframe for 0.5?

I will gladly step in and help implement the changes you proposed if that helps speed up the process :-)

[rstml]

Any changes in future version will provide upgrade path. For data model changes we will probably maintain backward compatibility for some time to ensure smooth upgrade. We are using it in production ourselves and this is definitely priority.

Thanks for offering help! 0.5 shaping up pretty quickly but no timelines yet. I'll ping this pull request with related task.

[ghost]

Hi,

any updates on this PR?