Recently in Cyrus IMAP Category

SOGo and Cyrus IMAP: 2.4 works well.

| | TrackBacks (0)
sogocyrus.png
The open source group-ware suite SOGo can be married happily together with Cyrus IMAP, using the Version 2.4.x of the powerful IMAP server.

The reason lies in the handling of the very important \Seen-Flag.

In Cyrus 2.3.x, you had to type this in your imapd.conf file:

flushseenstate: 1
In case you're interested in Sun/Oracle Openstorage, Fiberchannel Enterprise technologies or just want to drink a glass of fine beer - drop a note.

May 19th I am at Frankfurt Int'l Airport (Sheraton Airport Hotel) and from May 20th to 30th I will be staying in Cameroon meeting friends.

For Cameroon I am interested in internet connections. Last time I set up Orange Wimax connections and I am particularly interested in Camtel and MTN's offerings.

You may contact me at +49 171 6522660.

Pascal

Important update to my ldap auxprop demo plugin

| | TrackBacks (0)
Due to a variable scope bug Cyrus timsieved() can crash. The issue is resolved in version 1.2.2.

In case you are using this plugin, please update your installation. The source package can be found here:

giengerldap_auxprop-1.2.2.tar.gz

The md5 checksum is 636e8261198ea69372058e858fc496e9.

You will need Cyrus SASL 2.1.23 to compile it! Please update from 2.1.22 to 2.1.23!



TSM (Tivoli) and compressed ZFS with Cyrus IMAP

| | TrackBacks (0)
Just for the numbers:

Our tsm backup job scans 31 million files in 3 hours, including 300,000 deletes and 25,000 new files backed up:

01/03/09   23:38:48 --- SCHEDULEREC STATUS BEGIN
01/03/09   23:38:48 Total number of objects inspected: 31,305,555
01/03/09   23:38:48 Total number of objects backed up:   25,347
01/03/09   23:38:48 Total number of objects updated:          0
01/03/09   23:38:48 Total number of objects rebound:          0
01/03/09   23:38:48 Total number of objects deleted:          0
01/03/09   23:38:48 Total number of objects expired:    308,991
01/03/09   23:38:48 Total number of objects failed:           0
01/03/09   23:38:48 Total number of bytes transferred:    2.56 GB
01/03/09   23:38:48 Data transfer time:                  280.87 sec
01/03/09   23:38:48 Network data transfer rate:        9,594.01 KB/sec
01/03/09   23:38:48 Aggregate data transfer rate:        209.05 KB/sec
01/03/09   23:38:48 Objects compressed by:                   15%
01/03/09   23:38:48 Elapsed processing time:           03:34:49
01/03/09   23:38:48 --- SCHEDULEREC STATUS END
01/03/09   23:38:48 --- SCHEDULEREC OBJECT END SA_SO_MAIL 01/03/09   20:00:00
01/03/09   23:38:48
Executing Operating System command or script:
   /mail/bin/pr_backupsnapshot_off


This fantastical performance was achieved after activating GZIP compression on the mail zfs pool and copying all mail files to compressed format.

The machine is a 2xOpteron Dual Core with 20 GB of memory and 1200 mail users online connected.

Machine load did not go up noticeably. The mail pool is a zfs mirror with two SAN pools (7 TB each resulting in 7 TB mail capacity).

Tivoli Compression is set to on, and we use encryption to our TSM server.
We are using TSM client 5.5.1.0 for Solaris x86:

# dsmc
IBM Tivoli Storage Manager
Command Line Backup/Archive Client Interface
  Client Version 5, Release 5, Level 1.0 
  Client date/time: 01/08/09   09:40:56
(c) Copyright by IBM Corporation and other(s) 1990, 2008. All Rights Reserved.

Using that TSM client

memoryefficientbackup   yes
is very important in dsm.sys, otherwise dsmc will crash after having used 4 GB of RAM (it is a 32 bit executable, there is no 64 bit version for Solaris x86 available from IBM yet - in contrast to the SPARC 64 bit-Version).


Obsolete things can be nice. They can be useful. These "cmusaslsecret"-Thingies are.
But let's start from the beginning. You may however start to play with them!

You are using Cyrus SASL for authentification purposes? Supposedly with the Cyrus IMAP server or Postfix (the two major applications using Cyrus SASL)? Then you'll know the problem: You may use saslauthd(8) to connect to an authentication database as Unix PAM or LDAP. Problem: You will only be able to use plaintext logins. Why?

If using saslauthd, the SASL-enabled server gets the password given by the user/client, and asks saslauthd whether it is correct or not.

Plaintext authentication can be sufficient when using it with SSL/TLS. In IMAP environments this may be a good solution if all mail clients can be configured in that way. With SMTP you will have a problem. Many Mailservers being able to initiate authenticated outgoing SMTP connections do not use SSL/TLS. Many of them want to use CRAM-MD5 as it is defined in many standard drafts.

[Update 2009/06/25] Due to a variable scope bug Cyrus timsieved() can crash. The issue is resolved in version 1.2.2.

In case you are using this plugin, please update your installation. The source package can be found here:

giengerldap_auxprop-1.2.2.tar.gz

The md5 checksum is 636e8261198ea69372058e858fc496e9.
You will need Cyrus SASL 2.1.23 to compile it! Please update your SASL 2.1.22 installation, a security fix has been introduced.



[Update] Version 1.2.1 now accesses any attribute requested by SASL, so the use of cmusaslsecret* is possible. In this version the parameter "gl_attribute" has been omitted.

Downloadlinks in this article have been corrected to get the 1.2.1 version.


NOOOOO! Don't talk about saslauthd(8). Customer wanted CRAM-MD5 and DIGEST-MD5 as authentication mechanism for his Postfix authenticated SMTP service. No, not just PLAIN. So saslauthd is out of the game.

Customer has a "mail password" in cleartext in his LDAP structure [Update: he is using cmusaslsecretCRAM-MD5 now] - especially for this kind of thing. Is it possible to use Postfix with that?

First - this is not a postfix issue. Customer had Postfix linked with cyrus sasl. Not a bad idea - but to use these LDAP entries you have to have an appropriate cyrus sasl auxprop. Why?

CRAM-MD5 and DIGEST-MD5 are shared secret algorithms. The server MUST know the cleartext password or the mechanisms' secrets in order to validate the answer sent by the client.

At the first look, I saw a "ldapdb" auxprop plugin which should just do that - and it failed because we did not have a SASL enabled OpenLDAP so ldapdb authentication failed. "*cmusaslsecretCRAM-MD5" (in the case of CRAM-MD5, replace it with DIGEST-MD5 when using DIGEST-MD5) and userPassword are requested from the sasl auxprop.

So here it goes - I had to write my own ldap auxprop. You may use it if you want. I will expain the way to write SASL auxprops in the next days to come, but for now - here is the source.

Use syslogd(8) to get debug messages (loglevel debug, facility auth).


Just a side note:

We are using ZFS across two SAN mirror pairs for our production mailserver. It is running Cyrus IMAP with 14000 users and 50000 mailboxes. The zpool has 8 Terabytes and the system experienced some latencies when accessing data on the zpools.

The first problem was easy to resolve
but still we had IMAP SELECT times greater 0,5 seconds on some accesses.

The machine was doing approx 2000 random read accesses per second (we have approx 1000 concurrent IMAP users including our webmail application which is heavily using IMAP commands).

The first step was to put more RAM for the ARC cache in the machine, which resulted in 100-200 random read accesses per second. Cyrus metadata files of logged in users are now mainly in RAM.

The second step was to disable the prefetching algorithm. I put that in /etc/system:

set zfs:zfs_prefetch_disable=1

And - hooray - SELECT times are under 0,1 seconds. The prefetch method seems not to be optimized for a highly random i/o environment.

December 2015

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    

About

This blog is owned by:

Pascal Gienger
J├Ągerstrasse 77
8406 Winterthur
Switzerland


Google+: Profile
YouTube Channel: pascalgienger