16 feb 2011

oodiff reloaded

Although there exists alread oodiff, it's a bit limited. What's about any format or image changes not reflected in the diff?

I had a oodiff before, but some day it stopped working. Now I renamed it to ooodiff.
Thanks to some notes from here I corrected it. This version should work well with subversion. Here's the code (PEP8/Pyflakes compatible):


#!/usr/bin/python
# Requires python-uno
# Code originally gotten from:
# http://people.warp.es/~xtor/blog/?p=166
# See also: (google for "openoffice uno compare documents")
# http://win32com.goermezer.de/content/view/193/274/
# http://nxsy.org/blog/447.html
# http://www-verimag.imag.fr/~moy/opendocument/


PORT = 2526
OOFFICE = 'ooffice'

# General imports
import filecmp
import os
import shutil
import sys
import time

# pyuno imports
import uno
from com.sun.star.connection import NoConnectException


class OOUnoConnectionError(Exception):
pass


def getRevision(filename, revision, save_as):
"""Export a revision of a document."""
os.system('svn export -r%s "%s" "%s"' % (revision, filename, save_as))
return save_as


def compare(filename, oldrev='', newrev='', force=False):
"""Compare two revisions of a document."""
if not oldrev:
oldrev = 'BASE'
newfilename = '%s%s%s%s' % (filename, '.' + oldrev,
'.' + (newrev or 'workingcopy'), '.diff.tmp')
if not newrev:
shutil.copyfile(filename, newfilename)
else:
getRevision(filename, newrev, newfilename)
oldfilename = '%s%s%s' % (filename, '.' + oldrev, '.tmp')
getRevision(filename, oldrev, oldfilename)
if not force and filecmp.cmp(newfilename, oldfilename):
print ("The files seem to be the same. "
"There are no changes. Use --force run the diff in OO.")
else:
oo = OO()
doc = oo.open(newfilename)
oo.compareCurrentDocument(doc, oldfilename)


class OO:
"""A small class to abstract OpenOffice application."""
def __init__(self, filename=''):
# start OO
print "Starting OO"
os.system("soffice -nodefault "
"'-accept=socket,host=localhost,port=%s;urp;'" % PORT)

# Get the uno component context from the PyUNO runtime
localctx = uno.getComponentContext()
# Create the UnoUrlResolver on the Python side.
resolver = localctx.ServiceManager.createInstanceWithContext(
"com.sun.star.bridge.UnoUrlResolver", localctx)

cnxstr = ("uno:socket,host=localhost,port=%s;urp;"
"StarOffice.ComponentContext")
cnxstr = cnxstr % PORT
# try to connect to OO
ctx = None
retries = 5
while retries:
retries -= 1
try:
ctx = resolver.resolve(cnxstr)
except NoConnectException:
time.sleep(1)
if not ctx:
raise OOUnoConnectionError("Can't connect to OpenOffice")

# Get the ServiceManager object
smgr = ctx.ServiceManager

# Create the Desktop instance
desktop = smgr.createInstance("com.sun.star.frame.Desktop")
# save objects
self.desktop = desktop
self.smgr = smgr
self.ctx = ctx
if filename:
self.open(filename)

def open(self, filename, **kwargs):
"""Open a file."""
print "Opening: %s" % filename
properties = []
for key, value in kwargs.items():
properties.append(self.getProperty(key, value))
properties = tuple(properties)
doc = self.desktop.loadComponentFromURL(
self.convertToURL(filename), "_blank", 0, properties)
return doc

def compareCurrentDocument(self, doc, filename):
"""Compare the current document to the specified one."""
print "Comparing to: %s" % filename

# Sometimes, after opening OO or loading a doc
# we've still not access to the dispatcher or current frame
# so we wait here a bit.
# Get the dispatcher
dispatcher = self.smgr.createInstance("com.sun.star.frame."
"DispatchHelper")

# Show tracked changes and compare documents
frame = doc.getCurrentController().getFrame()
property = self.getProperty('URL', self.convertToURL(filename))
dispatcher.executeDispatch(frame, ".uno:CompareDocuments", "", 0,
(property,))
property = self.getProperty("ShowTrackedChanges", True)
dispatcher.executeDispatch(frame, ".uno:ShowTrackedChanges", "", 0,
(property,))

def getProperty(name, value):
"""Read a OO property."""
prop = uno.createUnoStruct("com.sun.star.beans.PropertyValue")
prop.Name, prop.Value = name, value
return prop
getProperty = staticmethod(getProperty)

def convertToURL(filename):
"""Convert a local filename to URL required by OO."""
return uno.systemPathToFileUrl(os.path.abspath(filename))
convertToURL = staticmethod(convertToURL)


if __name__ == '__main__':
oldrev = newrev = ''
force = '--force' in sys.argv
filename = ''
for arg in sys.argv[1:]:
if arg.startswith('-r'):
if not ':' in arg:
arg += ':'
oldrev, newrev = arg[2:].split(':')
elif not arg.startswith('-'):
filename = arg
if filename:
compare(filename, oldrev, newrev, force)
else:
print """
Usage: oodiff [--force] [-r[:]]
: the file to comparte
--force: force comparison, although files seem the same
-r : same as subversion. If not specified, -rPREV assumed.

This command:
- may not work if OpenOffice.org is already running (because
it has to start with a listening port for UNO to work).

- will create some temporary files (*.tmp). You have delete them
manually when you are finished.
"""

21 sept 2010

Tomcat 6: Session replication for failover

Summary: there is a bug in tomcat 6.0.20 which inhibits tomcat to send multicasts between instances, failing to form the cluster and hence not replicate sessions.

For the current project, I have to cluster Alfresco 3.2r Enterprise. We have a mini 2-node cluster. Although hibernate L2 cache replication works correctly (you have to rename ehcache-custom.xml.sample.cluster, which was not totally clear after reading the documentation), I went for session replication, which the docs state as supported. Now, only the session replication was missing. (There seems to be a bug which makes session replication fail, but I had no time to verify it. Anyway, I wanted to go ahead and learn how to configure tomcat for session replication and fail-over.)

As I had not much idea of configuring tomcat, I picked up an existing tomcat 6.0.20 instance and a small session example. The I configured tomcat, following the session replication / cluster how-to. Finally, I copied the tomcat instance and changed any colliding ports.

But I was not able to make it work. I was looking into the log for any message about my instances following the cluster, but without luck. After trying other ports, reconfiguring the network to support multicast ping (icmp), googleing around, reading a lot of docs, etc. I found a email message (which I can't find anymore), suggesting that there is a bug in tomcat-6.0.20 not sending multicasts for cluster instance detection!

I downloaded immediately a new version (6.0.29) and configured the two instances. It worked at the first attempt.

I use Apache proxy_balancer to test the instances. Here goes my Apache config file:
<Location /balancer-manager>
SetHandler balancer-manager
</Location>

<Proxy balancer://ajpCluster>
BalancerMember ajp://localhost:8809 route=jvm1
BalancerMember ajp://localhost:8810 route=jvm2
</Proxy>

<Location /sessiontest>
ProxyPass balancer://ajpCluster/sessiontest stickysession=JSESSIONID nofailover=off
</Location>

<Location /favicon.ico>
ProxyPass balancer://ajpCluster/favicon.ico
</Location>

The "route" parameter of BalanceMember adds just its value to the session id. The /balancer-manager url helps you to debug the cluster, displaying if both instances accept requests, how may have been processed, and to enable or disable any instances. As we can see here, my tomcat instances are listening for AJP requests on ports 8809 and 8810.

Here goes the interesting part of my conf/server.xml of both (the have just different ports):
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>

<Channel className="org.apache.catalina.tribes.group.GroupChannel">

<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
ttl="15"
port="45564"
frequency="500"
dropTime="3000" />

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4200"
autoBind="100"
selectorTimeout="5000"
maxThreads="6" />

<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>

<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>

</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.htm;.*\.html;.*\.txt;" />

<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false" />

<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>

<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
...
</Engine>


Take into account that you just have to change all enabled connector (HTTP, HTTPS, AJP, SHUTDOWN) ports and the cluster message Receiver port (4200, in the example above) for the tomcat instances which work in the machine.

13 dic 2009

Install Oracle PDO PHP (php_oci) driver in Ubuntu Hardy (8.04)

Ubuntu Hardy comes with some PHP PDO driver like mysql, postgresql and sqlite. Trying to install Oracle driver with:
sudo pecl install php_oci

fails in my Ubuntu with the following error:
pear/PDO_OCI requires PHP extension "pdo" (version  >= 1.0)

Isn't "pdo" installed already? It is part of php5-common, but PEAR/PECL doesn't know about it. So let's go ahead with ignoring dependency checks:
sudo pecl install -n php_oci

If you haven't installed php5-dev you'll get the following error:
sh: phpize: not found
ERROR: `phpize` failed

So install php5-dev, if you didn't already do it. Now, running the former pecl could yield:
...configure: error:
You need to tell me where to find oracle SDK, or set ORACLE_HOME.

Isn't ORACLE_HOME already set? Running a
echo $ORACLE_HOME
should answer this doubt. In my environment, it's set, but sudo doesn't pass it to the command, let's try again:
sudo -E pecl install -n pdo_oci

configure: error: Cannot find php_pdo_driver.h

The configure script tries to find it in:
/usr/include/php/....

This directory tree is empty or non-existent (in my Ubuntu).
cd /usr/include
sudo rm -rf php
sudo ln -s /usr/include/php5 php


Now the install succeeds. We have to enable it in /etc/php5/conf.d. Create a file 'pdo_oci.ini' with the following content:
#config por PDO OCI Oracle
extension=pdo_oci.so


Let's test is with php interactive mode:

$ php -a
php > $dbh = new PDO('oci:dbname=xe', 'system', '<password>');
php > $sql = 'select * from dual';
php > foreach ($dbh->query($sql) as $row){
php { print_r($row);
php { }
Array
(
[DUMMY] => X
[0] => X
)
php > [Control-D]


If we create now a simple php page which tries to show some Oracle output we get a PHP error message (depending on how error reporting is configured in PHP, it may not be rendered in PHP page, but be present in the log):
PDOException: SQLSTATE[]: pdo_oci_handle_factory: OCI_INVALID_HANDLE (/tmp/pear/cache/PDO_OCI-1.0/oci_driver.c:463) in ... on line ...

The problem is that Apache has no access to the ORACLE_HOME environment variable. Just add it to /etc/apache2/envvars:
export ORACLE_HOME=<path to Oracle, mine is: /usr/lib/oracle/xe/app/oracle/product/10.2.0/server>


Now we should be done.

30 oct 2009

OIOSAML and Blackboard / WebCT Vista/CE 8.0

Summary


When installing OIOSAML, incompatibility issues arise. This blog entry provides detailed information about how to fix this problem and may be applicable to other applications that need to run with Sun Java 1.5 and updated versions of JAXP 1.3.

Introduction


Blackboard Vista / CE 8.0, formerly known as WebCT, uses Bea Weblogic 9.2 application server, including Sun Java 1.5. OIOSAML requires and provides an updated version of JAXP 1.3, which is part of Java 1.5 core library. To override internal libs in Java 1.5, you have to "endorse" the new libraries. This is correctly described in OIOSAML docs.

Symptoms


When starting the Weblogic server, the following error appears:
weblogic.management.ManagementException: [Management:141266]Parsing Failure in config.xml:
javax.xml.namespace.QName; local class incompatible:
stream classdesc serialVersionUID = 4418622981026545151, local class serialVersionUID = -9120448754896609940


How to fix the problem


You have to include the following option in JAVA_OPTIONS ($webct_domain_dir/customconfig/startup.properties):
-Dorg.apache.xml.namespace.QName.useCompatibleSerialVersionUID=1.0


The startup.properties already contains:
-Dcom.sun.xml.namespace.QName.useCompatibleSerialVersionUID=1.0


But the provided xerces library (2.9.1) has another property name as the one provided with Sun Java 1.5.

Detailed description


When Sun included JAXP 1.3 in Java 1.5, it changed the namespace in all source files. It also specified an explicit serialVersionUID for javax.xml.namespace.QName. In Java 1.5, the serialVersionUID was explicitly specified as 4418622981026545151 (0x3d521a30bc76fdff). But in previous versions of JAXP (xerces) this was implicitly calculated as : -9120448754896609940 ( 0x816da82dfc3bdd6c).

The already serialized data could not be deserialized. SUN corrected this error in next releases, introducing a compatibility flag

com.sun.xml.namespace.QName.useCompatibleSerialVersionUID
which having the value 1.0 switches to the Java 1.5 serial UID.

The Xerces project people picked up this change in newer version of xerces, but changed the flag name to
class="wiki">org.apache.xml.namespace.QName.useCompatibleSerialVersionUID


I recommend setting both of the settings. As the 'com.sun' options is already specified you just need to specify the 'org.apache' one:
JAVA_OPTIONS="$JAVA_OPTIONS -Dorg.apache.xml.namespace.QName.useCompatibleSerialVersionUID=1.0

20 nov 2007

Problems launching VirtualBox in headless mode with ssh

I recently installed a VirtualBox image to run remotely using ssh. My command was:

VBoxVRDP -startvm "GuestOS" >~/VBoxVRDP.log 2>&1 &

inside the ssh session.

Whenever closed the ssh session, the virtual machine was killed. I thought it was a problem of the terminal, so I tried out screen and dtach. But the problem was remaining. Googling around, I finally found:
http://www.virtualbox.de/ticket/722
In effect, I had shared clipboard enabled. After I disabled it, I could finally start VirtualBox using:
ssh -x zope@192.168.11.201 'VBoxVRDP -startvm "GuestOS" >~/VBoxVRDP.log 2>&1 &'

Now, I'm a bit happier.
Cheers.

14 nov 2007

NTLM Authentication in Django

Recently I had to integrate NTLM Intranet authentication into a Django application. The first problem was to get mod_ntlm [1] to work in Ubuntu Feisty [2]. After this was done I had to configure my Samba as a Primary Domain Controller (PDC) and add my vmplayer WinXP instance to that domain [3] [4].

After that was done, I wrote a Django authentication backend based on [5].

What I did:
  1. Add a link to my customized registration/login.html page which points to a special location which is protected by mod_ntlm, e.g.:
    <a href="./remote_user/?next={{next}}">Intranet authentication</a>

  2. Configure this location in Apache2 for mod_ntlm:


    #NTLM Auth
    AuthName NTAuth
    AuthType NTLM
    NTLMAuth on
    NTLMAuthoritative on
    NTLMDomain DOMAIN
    NTLMServer pdc.sercer
    NTLMBasicAuth off
    # NTLMBasicRealm SISAM
    NTLMLockfile /tmp/_my.lck
    # NTLMBackup
    Require valid-user
    # Satisfy all

    Here, I suppose that all django mod_python config is already included in /.

  3. Write the authentication backend and a view that captures the "REMOTE_USER" environment variable, authenticates and logs in the user. Here is my "remoteuser.py":

    """User auth based on REMOTE_USER.
    To make it work you need:
    - add RemoteUserAuthBackend en settings.py, en AUTHENTICATION_BACKENDS
    - add ('/login/remote_user/', 'sisamapp.auth.remoteuser.remote_user_login') to your urls.py
    - enable the apache module (e.g. mod_ntlm)
    - configure Apache /login/remote_user/, e.g. for mod_ntlm:

    #NTLM Auth
    AuthName NTAuth
    AuthType NTLM
    NTLMAuth on
    NTLMAuthoritative on
    NTLMDomain DOMAIN
    NTLMServer MACHINE or IP
    # NTLMBasicAuth off
    # NTLMBasicRealm SISAM
    NTLMLockfile /tmp/_my.lck
    # NTLMBackup
    Require valid-user
    # Satisfy all

    We suppose here that the / location has already all django stuff configured (i.e. PythonHandler)
    """
    from django.contrib.auth.models import User

    import sys
    log = sys.stderr.write

    # copied from http://code.djangoproject.com/attachment/ticket/689/remote_user_2.diff
    from django.contrib.auth.backends import ModelBackend
    class RemoteUserAuthBackend(ModelBackend):
    def authenticate(self, **credentials):
    """
    Authenticate user - RemoteUserAuth middleware passes REMOTE_USER
    as username. password param is not used, just added in case :)
    """
    try:
    type = credentials['type']
    if type == "remote_user":
    username = credentials['username']
    except:
    username = None
    if not username:
    return None
    user = None
    try:
    user = User.objects.get(username=username)
    except User.DoesNotExist:
    raise User.DoesNotExist, _T('User %s not configured in this application.') % username
    return user

    class NoRemoteUserInfoAvailable(Exception):
    pass


    from django.http import HttpResponseRedirect
    from django.shortcuts import render_to_response
    from django.template import RequestContext
    import re
    from django.utils.translation import ugettext as _T
    def render_notice(request, errornote, msg):
    return render_to_response('registration/notice.html',
    {'errornote': errornote, 'msg': msg },
    context_instance = RequestContext(request))

    def remote_user_login(request):
    error = """
    remote_user_login requires Django authentication middleware to be installed. (Include in MIDDLEWARE_CLASSES setting 'django.contrib.auth.middleware.AuthenticationMiddleware'.
    """
    msg = _T('Use the __standard login form__ to provide alternative credentials.')
    msg = re.sub('__(.*)__',r'<a href="../?next=%s">\1</a>' % request.GET.get('next',''), msg, re.UNICODE)
    try:
    username = request.META['REMOTE_USER']
    log("Got REMOTE_USER=%s\n" % username)
    except:
    return render_notice(request,
    errornote=_T('Server does not provide REMOTE_USER.'),
    msg=msg)
    if not username:
    return render_notice(request,
    errornote=_T('Could not get your credentials. Are you accessing from anywhere outside the domain or a browser that does not support intranet authentication?'),
    msg=msg)
    from django.contrib.auth import authenticate, login
    # AuthenticationMiddleware is required to create request.user
    assert hasattr(request, 'user'), self.error
    if request.user.is_anonymous():
    log("Request is anonymous. Trying to authenticate user %s\n" % username)
    try:
    user = authenticate(username=username, type="remote_user")
    except:
    user = None
    log("User is %s\n" % user)
    if user is not None:
    request.user = user # set request.user to the authenticated user
    login(request, user) # auto-login the user to Django
    return HttpResponseRedirect(request.GET.get('next','/'))
    return render_notice(request,
    errornote=_T('Your username (%s) is not registered here.') % username,
    msg=msg)
    # user is already authenticated, should logout first
    msg = _T('You have to logout first using __this link__ before logging in again.')
    msg = re.sub('__(.*)__',r'<a href="../../logout/">\1</a>', msg, re.UNICODE)
    return render_notice(request,
    errornote=_T('You are already authenticated.'),
    msg=msg)

    Some notes here:

    • remote_user 'authenticate' uses explicitly another signature as ModelBackend 'authenticate', i.e. it needs the 'type' argument. If you used the same signature (username, password) there is a possibility that a user authenticates without any password!
    • In my configuration, when I acces with Firefox/Linux /login/remote_user/, a browser authentication dialog pops up. I was not able to get rid of it.

    Links:
  1. http://modntlm.sourceforge.net/
  2. http://erny-rev.blogspot.com/2007/11/compiling-modntlm-for-apache2-in-ubuntu.html
  3. http://geeklab.wikidot.com/samba-pdc
  4. http://us1.samba.org/samba/docs/man/Samba-HOWTO-Collection/domain-member.html#machine-trust-accounts
  5. http://code.djangoproject.com/attachment/ticket/689/remote_user_2.diff

13 nov 2007

Compiling mod_ntlm for Apache2 in Ubuntu Feisty


  1. Do a checkout
    svn co https://modntlm.svn.sourceforge.net/svnroot/modntlm/trunk/mod_ntlm2

  2. install apache-developer files:
    sudo aptitude install apache2-prefork-devel 

  3. cd into mod_ntlm2 and edit Makefile:
    APXS=apxs2
    APACHECTL=apache2ctl

  4. edit mod_ntlm.c:
    replace
    apr_pool_sub_make(&sp,p,NULL);
    with
    apr_pool_create_ex(&sp,p,NULL,NULL);

    As of this post, apr_pool_sub_make is deprecated and was removed. Use apr_pool_create_ex instead.

  5. make (there's a warning about log function). Do not 'make install' here. (it doesn't find mod_ntlm.so and it adds LoadModule to httpd.conf instead of using new-style module loading mechanism.

  6. copy it to apache2 modules dir:
    sudo cp .libs/mod_ntlm.so /usr/lib/apache2/modules/

  7. create file /etc/apache2/mods-available with sudo:
    LoadModule ntlm_module /usr/lib/apache2/modules/mod_ntlm.so

  8. configure some location or directory which needs authentication:
           #NTLM Auth
    AuthName NTAuth
    AuthType NTLM
    NTLMAuth on
    NTLMAuthoritative on
    NTLMDomain DOMAIN
    NTLMServer your.dc.here # or IP
    NTLMLockfile /tmp/_my.lck
    # NTLMBackup your.dc-backup.here
    require valid-user

  9. reload apache2:
    sudo /etc/init.d/apache2 force-reload

See: