Moving home

I've moved house to aelfric.co.uk

Solar panels - the first quarter

So we've had our solar panels installed for three months and inspired by my friend SomeBeans I thought I'd write a blog post to comment on the experience.

We had our panels installed by Joju (a link to us as a case study here) in early June - just about when the sunny weather stopped and we started the least sunny summer for 18 years. Our estimated generation is 2,300 kWh each year so a fraction over 900 kWh for what should be the sunniest months of the year is slightly disappointing - hopefully better weather would lead to another 10% or so but then when do we ever have a sunny summer....

Because of the shape and configuration of our roof we ended up with installing some high end Sanyo panels which gives us a total installed capacity of 2.88 kWh - basically these panels are a bit smaller so we could fit more on the back roof and if we wanted to put some cheaper ones on the other then we would need two inverters which would be more expensive anyway.
(The highest number I've spotted being generated is 2,798W)

Our roof faces roughly SW which means that the panels don't start working properly until after 10 in the morning but then keep going after that.

The installation cost about £13,000 and we've just had a cheque for about £400 based on 900kWh. Of course that doesn't take into account what is saved in consumption
(For ease of comparison we pay roughly 20p per kWh for first 225kWh per quarter followed by 10p - interestingly we get paid 3.1p for each kWh for the export tariff)
The estimate is half used so on that basis it's a further £45 - I suspect it's more than that however as we are making an effort to do as much as possible when it's sunny.
All this is tax free.

If we assume an interest rate of 3% over 20 years then our £13,000 would turn into £23,479.45 (that makes no allowance for tax)

Reading Date	Reading	Incremental Value kWh	End Date	End Reading
05-09-11	874.6	38	12-09-11	913
29-08-11	818.8	56	05-09-11	875
22-08-11	760.3	58	29-08-11	819
15-08-11	702.1	58	22-08-11	760
08-08-11	639.1	63	15-08-11	702
01-08-11	565.9	73	08-08-11	639
25-07-11	491.1	75	01-08-11	566
18-07-11	426.6	64	25-07-11	491
11-07-11	350.6	76	18-07-11	427
04-07-11	277.9	73	11-07-11	351
27-06-11	214.5	63	04-07-11	278
20-06-11	132.7	82	27-06-11	214
13-06-11	53.1	80	20-06-11	133
10-06-11	31.2	22	13-06-11	53

Changes to our consumption

It's quite hard to tell because we had a faulty meter last year but saving a third seems like a good guess.

As we don't have a smart meter (even though it's less than a year old) the energy company assumes that we consume 50% of our generated power and export the other 50% - this means that we were paid for 450 kWh. What this assumption means is that when we're generating anything we use is effectively free!

Bill period	Number of days	Total	Daily Average
22/07/11 - 10/09/11	51	343	7
06/06/11 - 21/07/11	46	275	6
22/04/11 - 05/06/11*	45	407	9
31/01/11 - 21/04/11	81	916	11

* We were on holiday for a week during this period

Alfresco debugger in a VirtualBox VM

How to run the Alfresco Server Side debugger when running in an Ubuntu Virtual Box VM

Edit /opt/alfresco-3.4.d/tomcat/scripts/ctl.sh to add:
DISPLAY=:0.0
export DISPLAY

The other trick is to make sure that you type 'xhost +'

You will need to do this in a shell window AND in the shell from which you restart alfresco


The Web Script /alfresco/service/api/javascript/debugger has responded with a status of 500 - Internal Error.

500 Description: An error inside the HTTP server which prevented it from fulfilling the request.

Message: 08090007 Wrapped Exception (with status template): Could not initialize class sun.awt.X11GraphicsEnvironment

Exception: java.lang.NoClassDefFoundError - Could not initialize class sun.awt.X11GraphicsEnvironment

Screen resolution for ubuntu 11 in virtualbox 4

Thanks to this link:
http://www.webupd8.org/2010/12/how-to-test-ubuntu-1104-with-unity-in.html

In a nutshell:
sudo apt-get update
sudo apt-get install virtualbox-ose-guest-utils

(Also enable 3D acceleration)

CAS, Alfresco and WebDAV

Having successfully configured Alfresco (and Share) to authenticate using CAS - it's documented here

The next challenge is to try and work out how to get this authentication working with webdav
Update:

One way to do this this is to bypass CAS and authenticate against the underlying CAS datastore directly by adding a new authentication component.

Of course this means that you are not using CAS but then as you are likely to be accessing WebDAV outside of the browser the single sign on capabilities are not particularly relevant

We use Drupal as our underlying CAS data store so there's a bit of custom code here - you may be able to just configure the authentication chain if you're using a different method e.g. LDAP

So in alfresco-global.properties add to the authentication chain:
authentication.chain=cas:external,localDrupal:drupal

While you're there set up some database connection properties e.g.

drupal.db.driver=org.gjt.mm.mysql.Driver
drupal.db.username=drupal
drupal.db.password=drupal
drupal.db.url=jdbc:mysql://localhost:3306/drupal6

I'm not going to cover ensuring that you have access to the mysql database here. If it's on a different machine you'll need to configure MySQL as well as any firewall rules.

Next it's time to tell Alfresco about the new component that you are creating so create the directory alfresco/subsystems/Authentication/drupal and add the following files:

drupal-authentication-context.xml

<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xsi:schemalocation="http://www.springframework.org/schema/beans
          http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
    
    <bean class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" id="drupalDataSource">
        <property name="driverClassName" value="${drupal.db.driver}">
        <property name="url" value="${drupal.db.url}">
        <property name="username" value="${drupal.db.username}">
        <property name="password" value="${drupal.db.password}">
        <property name="validationQuery" value="SELECT 1">
        <property name="testOnBorrow" value="true">
        <property name="defaultAutoCommit" value="false">
        <property name="maxWait" value="5000">
    </property></property></property></property></property></property></property></property></bean>

    
    <bean class="mypackage.authentication.DrupalAuthenticationComponentImpl" id="drupalAuthenticationComponent" parent="authenticationComponentBase">
        <property name="dataSource" ref="drupalDataSource">
        <property name="nodeService">
           <ref bean="nodeService">
        </ref></property>
        <property name="personService">
           <ref bean="personService">
        </ref></property>
        <property name="transactionService">
           <ref bean="transactionService">
        </ref></property>
    </property></bean>

    
    <bean class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean" id="AuthenticationComponent">
        <property name="proxyInterfaces">
            <value>org.alfresco.repo.security.authentication.AuthenticationComponent</value>
        </property>
        <property name="transactionManager">
            <ref bean="transactionManager">
        </ref></property>
        <property name="target">
            <ref bean="drupalAuthenticationComponent">
        </ref></property>
        <property name="transactionAttributes">
            <props>
                <prop key="*">${server.transaction.mode.default}</prop>
            </props>
        </property>
    </bean>

    
    <bean class="org.alfresco.repo.security.authentication.AuthenticationServiceImpl" id="localAuthenticationService">
        <property name="ticketComponent">
            <ref bean="ticketComponent">
        </ref></property>
        <property name="authenticationComponent">
            <ref bean="drupalAuthenticationComponent">
        </ref></property>
        <property name="sysAdminParams">
            <ref bean="sysAdminParams">
        </ref></property>
    </bean>

</beans>

drupal-authentication.properties

drupal.db.driver=org.gjt.mm.mysql.Driver
drupal.db.username=drupal
drupal.db.password=drupal
drupal.db.url=jdbc:mysql://localhost:3306/drupal6

Then create the bean that is referenced from the config

package mypackage.cms.authentication;

import org.alfresco.repo.management.subsystems.ActivateableBean;
import org.alfresco.repo.security.authentication.AbstractAuthenticationComponent;
import org.alfresco.repo.security.authentication.AuthenticationException;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.dao.DataAccessException;
import org.springframework.jdbc.core.JdbcTemplate;

import javax.sql.DataSource;
import java.security.MessageDigest;

/**
 * Authenticates a user by Drupal.
 * 
* The authentication is done against the database table USERS and the
* column NAME (username) and column PASS (MD5 password)
* 
 * Tested with Drupal 6
 *
 */
public class DrupalAuthenticationComponentImpl extends AbstractAuthenticationComponent implements ActivateableBean {
    public static final String GET_USER_PWD_SQL = "SELECT PASS FROM USERS WHERE NAME=?";

    private final Log logger = LogFactory.getLog(getClass());
    private JdbcTemplate m_jdbcTemplate;
    private boolean m_active = true;

    public DrupalAuthenticationComponentImpl() {
        super();
    }

    public void setActive(boolean active) {
        m_active = active;
    }

    public void setDataSource(DataSource dataSource) {
        m_jdbcTemplate = new JdbcTemplate(dataSource);
    }

    /*
    * (non-Javadoc)
    * @see org.alfresco.repo.management.subsystems.ActivateableBean#isActive()
    */
    public boolean isActive() {
        return m_active;
    }

    /**
     * Authenticate against the Drupal database
     *
     * @param userName the username to authenticate
     * @param password the password to authenticate (passed in as plain text)
     * @throws AuthenticationException if authentication failed
     */
    @Override
    protected void authenticateImpl(String userName, char[] password) throws AuthenticationException {
        String userPwd = new String(password);

        // Generate an MD5 hash for the password as that is what we get back from Drupal
        // Get the value as hex
        String userPwdMd5 = DigestUtils.md5Hex(userPwd);

        if (logger.isDebugEnabled()) {
            logger.debug("About to authenticate user: " + userName + " with MD5 password: " + userPwdMd5);
        }

        try {
            String drupalPwdMd5 = m_jdbcTemplate.queryForObject(GET_USER_PWD_SQL, new Object[]{userName}, String.class);

            if (logger.isDebugEnabled()) {
                logger.debug("Got MD5 password from Drupal database: " + drupalPwdMd5);
            }

            if (StringUtils.isNotBlank(drupalPwdMd5)) {
                if (MessageDigest.isEqual(userPwdMd5.getBytes(), drupalPwdMd5.getBytes())) {
                    // Authentication has been successful.
                    // Set the current user, they are now authenticated.
                    setCurrentUser(userName);
                } else {
                    throw new AuthenticationException("Access denied for user: " + userName +
                            ", incorrect password provided.");
                }
            } else {
                throw new AuthenticationException(
                        "Password in Drupal database is blank, empty, or null for user: " + userName);
            }
        } catch (DataAccessException dae) {
            throw new AuthenticationException(
                    "Error getting password from Drupal database for user: " + userName +
                            ", user may not exist in the Drupal database", dae);
        }
    }

    @Override
    protected boolean implementationAllowsGuestLogin() {
        return true;
    }
}

XML and relational databases

A problem I seem to keep to getting asked for help with is how to persist XML data to a relational database. (I’ve now been asked to help on at least four different projects where this has become something of an issue/blocker although strangely I’ve never actually needed to do this myself)

The context of this is usually services – I’ve seen this in REST and SOAP based services – not a debate I’m going to enter into here.

Note that this post is concerned with traditional relational databases and does not cover using the XRX (XForms, REST, Xquery) model or XML databases such as MarkLogic or eXist.

The first thing I recommend doing is to ensure that you have an XML schema. I think that it’s good practice and makes you think about the design of your XML structures. If you’re more familiar with designing databases or programmatic data structures then this will help you think about your XML in the same way and save you problems later on. It’s all too easy to end up with the same element name used for different things or common data structures with slightly different names if you just use well-formed XML.

What’s the best way to do this? Well as usual it depends…

Java driven approach

Here the starting point is your java beans.
You can use annotations for both JPA, to define the object relational mapping, and JAXB to define the object XML mapping.
http://www.oracle.com/technetwork/articles/marx-jse6-090753.html

XML driven approach

Here the XML is the primary component. Information about the structure of the data is maintained in the schema together with the JPA annotations.
Using the schema annotations it is possible to customize the way that the java beans are generated.
This is a nice approach because all the information about the structure of the data is held in the same place and everything can be easily regenerated and you don’t have to worry about losing any changes made to the generated classes.
You will, of course, need to ensure that your database schema stays in sync with the JPA definitions.
Option #1 - Hyperjaxb3

http://confluence.highsource.org/display/HJ3/Home

Option #2 - Use Dali to map your POJOs to Database (JPA)

The Eclipse Dali tool provides tooling to easily map your POJOs to a relational database using JPA:

http://www.eclipse.org/webtools/dali/

Option #3 - Use EclipseLink

EclipseLink provides both JPA and JAXB implementations. The JAXB implementation (MOXy) contains extensions specifically for handling JPA entities:

http://wiki.eclipse.org/EclipseLink/Examples/MOXy/JPA
Other links
http://www.slideshare.net/shaunmsmith/restful-services-with-jaxb-and-jpa
http://blog.hma-info.de/2008/05/15/hyperjaxb-3-the-fastest-way-from-xml-to-db/

http://blogs.bytecode.com.au/glen/2010/07/29/from-wsdl-to-jaxb-to-jpa-with-a-single-schema--adventures-in-hyperjaxb3.html

XML driven approach - a variation

An alternative approach to annotating the XML schema is to use aspect orientated programming and place the JPA annotations in the AspectJ files (a bit like how Spring Roo works)

Database driven approach

In this approach the database is the most important component in the system – this is a good approach if you’ve got an existing, stable database and want to be able to quickly add XML capabilities e.g. to provide a service based interface

Create JPA beans

There are a number of ways to do this – you can of course write the beans by hand but it’s easier to generate them. You can do this using Spring Roo (see earlier blog post) or Eclipse Dali

Mapping JPA to XML

The obvious way to map between XML and Java beans is to use JAXB.
One approach is to generate the JAXB beans and write a custom mapping to the JPA beans however there is a better way.
If you use the MOXy JAXB implementation then there are some extensions which you can use :
• XPath Based Mapping
• JPA Entities to XML - Bidirectional Relationships
These extensions allow you to use annotations on your JPA beans to describe your JAXB mappings.
N.B. If you regenerate your JPA beans then these annotations will be lost – a major disadvantage to this approach

A detailed example of how to do this is at:
http://bdoughan.blogspot.com/2010/08/creating-restful-web-service-part-15.html

The common approach

I’ve called in this because, in my experience, this is what most java coders do. In fact this covers several different approaches but at heart they are very similar.

Parse the XML into java beans

Lots of ways to do this – JAXB, using DOM or SAX by hand, if you’ve a well written WSDL file for a SOAP service then this will be part of the generated code.

Persist the java beans

Again lots of ways to do this – hand written JDBC calls, Hibernate…

The simple approach

This is a simple approach which might be sufficient if performance is not too much of an issue.
This could be a good approach if you’ve got some relatively complex XML, which you want to store, but are not interested in much of the actual content of the XML.
It is easy enough to store XML documents as BLOB and all the signification databases now support XPATH querying of the data (Check your database for specifics of how to do this e.g. MySQL at http://dev.mysql.com/tech-resources/articles/xml-in-mysql5.1-6.0.html#xml-5.1-xpath)
If you are using this approach then it’s probably a good idea to use stored procedures so that if you need to migrate to a more structured data model then this can be accomplished with minor application changes.
This approach can be enhanced by extracting specific elements into structured columns so that if there is a particular element that you want to query you can do it in the traditional manner.

ETL Tools

If you’ve got the tools and the experience in using them then ETL tools can provide a graphically driven interface to produce the mapping between the XML and the database schema. Talend, Oracle Data Integrator, IBM Data Stage, Informatica etc all provide this and can expose service endpoints for interactions. If you’ve got the tools and expertise in using them then it’s worth considering this approach.

I think this post is probably long enough for now…

Orbeon upgrade

Notes on upgrading from Orbeon Forms 3.8 to 3.9

Need to change theme.xsl as follows:

From:
<!-- Copy body attributes -->
<xsl:apply-templates select="/xhtml:html/xhtml:body/@*"/>

<xsl:apply-templates select="/xhtml:html/xhtml:body/node()"/>

<xsl:for-each select="/xhtml:html/xhtml:head/xhtml:script">
<xsl:element name="xhtml:{local-name()}" namespace="{namespace-uri()}">
<xsl:apply-templates select="@*|node()"/>
</xsl:element>
</xsl:for-each>

To:
<!-- Body -->
<xsl:apply-templates select="/xhtml:html/xhtml:body"/>
<!-- Handle post-body scripts if present. They can be placed here by oxf:resources-aggregator -->
<xsl:apply-templates select="/xhtml:html/xhtml:script"/>

Use of xxforms:get-remote-user (xforms:setvalue)

From:
<xforms:setvalue ref="instance('ins-current-user')/name" value="xxforms:get-remote-user()"/>

To:
<xforms:setvalue ref="instance('ins-current-user')/name/text()" value="xxforms:get-remote-user()"/>

Ongoing is the use of XBL - the map component in particular
(Fixed in the final release)