CEDANIEL.COM

This post applies to Microsoft Office SharePoint Server 2007, WSS 3.0, Adobe LiveCycle, and PDF Forms

Scenario: Many organizations use Adobe PDF forms for employees to submit organizational forms for various business processes. It’s easy to configure these forms to submit the data in an e-mail message or send it to a SQL Server database. Submitting data is usually not the only thing required in the business process of the form. Once the data gets submitted via e-mail or database, there is usually some type of workflow involved in the business process. Adobe has a product called Adobe LiveCylcle ES that has a workflow engine that can be used to process information within an organization via Adobe Forms. Adobe LiveCycle ES is an incredible product but it is very expensive and you may not have it available within your organization. As an alternative, it is possible to post data from an Adobe PDF form and submit it to a SharePoint List. It is actually a very easy process. So….. How do you solve this?

Solution: First of all, I’m not sure why – but there is not much to find out there on this problem. I’m not sure exactly why. If you learn this technique, you could easily create very dynamic forms using Adobe LiveCycle Designer that can meet difficult requirements and leverage the use of SharePoint for data storage and Workflow processes. This Adobe blog post helped me with this process: Click Here to Read an Adobe Blog Post

You could easily read the post from the link above, so I’m going to explain it to you from my perspective.

Basically, to get Adobe PDF forms to communicate with a SharePoint List you need a 3 layer architecture.

LAYER 1: Adobe Form with a Submit button that is configured to submit data as XML and send the XML data to the http path of an .ashx file that is stored in the 12 hive (or another location) on a SharePoint server

LAYER 2: An .ashx file is created using Visual Studio  (or any text editor) and stored in the 12 hive (C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\Whatever_You_Want_To_Call_This_Subdirectory) of the SharePoint Server.

LAYER 3: A Custom SharePoint List is created that has fields that match the fields of the Adobe Form.

Now I will explain each of these 3 Layers in more detail.

LAYER 1 (The Adobe Form):

First, you will want to use Adobe LiveCycle Designer to design the fields for your form. Just create a simple Text field called “Title” and a “Submit” button on the form. Then configure the Submit button to submit the data as XML. When configuring the submit button for XML, it will ask you for a desination URL to post your data. This is where you will paste the http: path from SharePoint to locate the .ashx file.  If you stored the file in a subdirectory of the LAYOUTS directory of the 12 hive, it would be something like:  http://servername/sites/sitecollectionname/_layouts/whatever_subdirectory/whatever.ashx

Now your form is prepared for data submission to a SharePoint List using an HttpHandler. Now lets keep going here.

LAYER 2 (The HTTPHandler – .ashx file)

You will need to create an HTTPHandler (.ashx) file with Visual Studio to handle accepting XML data from the Adobe PDF form and send it to the SharePoint List. The HTTPHandler will basically be used as the “middle man” to handle the exchange of the data. The file can be stored in the 12 hive of the SharePoint server so that it can be visible by all site collections within the SharePoint Farm. To use the 12 hive, you will need to create a subdirectory in the following path:  C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\LAYOUTS\Whatever_Subdirectory

You could easily just store .ashx file in the LAYOUTS directory, but its best practice not to do this so that you don’t risk replacing a file with the same name which could cause your SharePoint farm to become unstable.

The easiest way to create an .ashx file in Visual Studio is just to create a Web Application project using C#/ASP.NET. Then add a “New Item” as a basical HTML page. Then just rename the HTML Page with a .ashx file extension. There are other ways but this is easy enough.

Now let’s take a look at the anatomy of this .ashx file. First of all, you will be programming against SharePoint Services to get the data to the SharePoint list. So let’s look at a sample ASP.NET/C# script:

CODE ::
<%@ Assembly Name=”Microsoft.SharePoint,Version=12.0.0.0,Culture=neutral,PublicKeyToken=71e9bce111e9429c” %>
<%@ WebHandler Language=”C#” Class=”put_data_in_sp_list” %>

using System;
using System.Web;
using Microsoft.SharePoint;
using System.Xml;

public class put_data_in_sp_list : IHttpHandler {

public void ProcessRequest (HttpContext context) {
SPSite site = SPContext.Current.Site;
SPWeb web = site.OpenWeb();

try
{
string rawXML = “”;
XmlTextReader reader = new XmlTextReader(context.Request.InputStream);
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.Load(reader);
string _xmlString = xmlDoc.InnerXml;
System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
string _fileTime = DateTime.Now.ToFileTime().ToString();

byte[] docAsBytes = encoding.GetBytes(_xmlString);

//Insert Document
web.AllowUnsafeUpdates = true;
// Here is the name of your Custom SharePoint List
SPList list = web.Lists["TestList"];
SPListItem item = list.Items.Add();

// Here is the name of your SharePoint Field that you will be sending the data from the first field of your Adobe Form
item["Title"] = xmlDoc.GetElementsByTagName(”Title”).Item(0).InnerText;
// Now Update the SharePoint List with the Data
item.Update();

}
catch (Exception ex)
{
context.Response.Write(ex.Message);
}

}
// The HTTPHandler must have these lines
public bool IsReusable {
get {
return false;
}
}

}

:: END OF CODE

Notice that this script is using a single text field in an Adobe form then sending the XML data to a Custom SharePoint list called “TestList” in the “Title” field using SharePoint Services. If you have ever programmed C# scripts against SharePoint Services, you can see how this technique is EXTREMELY powerful. Since the referenced class above is “put_data_in_sp_list”, then obviously your .ashx file will be called put_data_in_sp_list.ashx .

LAYER 3 (Custom SharePoint List):

The final piece of this architecture is the Custom SharePoint List. If you are working off of the example above, obviously you will need to create a Custom SharePoint List called “TestList” and just use the “Title” field.

Conclusion:

Now that you have all 3 layers in place, you can now Preview your Adobe PDF form in Adobe LiveCycle Designer and enter some text in the text field. Then hit the Submit button. Your data will now be sent to the HTTPHandler, then added as a new item in the SharePoint “TestList”.

Once you have successfully tested data going from the Adobe PDF form to the custom SharePoint List, you are now ready to create your custom workflow on the SharePoint list. Just use SharePoint Designer to create the workflow to meet your business process needs and set it to kick off when a new item is added to the “TestList”. Install the worklfow on the “TestList” and now you have an end-to-end business solution using an Adobe PDF form to submit data to a SharePoint list while leveraging the Workflow capabilities of SharePoint.

This post applies to Microsoft Office SharePoint Server 2007, WSS 3.0, SQL Server 2008 Integration Services, and MySQL.

Scenario: Many times as SharePoint lists grow to the 1000s of rows, it is more applicable to build your reporting system external of the SharePoint system. Data Views begin to be difficult to deal with and you may need to build a custom solution for optimal performance. In my case, I have a mixed shop of developers that have a variety of skills. When I lead a project with my team, I try to distribute the workload as best as I can so that all can contribute to the quality and completion of the project. One of my developers is extremely talented in the world of Linux, Apache, MySQL, and PHP (aka LAMP). I still believe that the LAMP stack is the best environment for web applications but lately the requirements of projects have been dominated by the world of SharePoint. SharePoint has become Microsoft’s hostile takeover environment. It seems they are wrapping all of their services underneath it. All stack examples I’ve seen lately seem to include SharePoint somewhere in the pipe.

So, knowing that IT Shops are buying into SharePoint (hardcore) and executives are getting used to its look and feel.  SO… what do we do with our highly skilled LAMP developers? They cannot be left out of the project. In my opinion, we need to at least give them the reporting portion of the project. However, according to this technique that I’ll describe in the solution below it is possible to build a LAMP solution that can make a transaction to a SharePoint list. I have not fully tested this solution yet – but in theory it should work. So…….. how do we do this?

Solution: In summary: What you will be doing is using Microsoft SQL Server 2008 Integration Services to build a package that will take data from a SharePoint List to MySQL. This solution is possible using Microsoft SQL Server 2005 but it is much easier using the 2008 version. SSIS 2008 comes with an ADO.NET destination component for the ETL engine Data Flow. The ADO.NET component made it much easier to set up the connection and send the data to MySQL.

First you need to setup an ODBC connection on your development box that uses an account that can successfully connect to your MySQL database. Go to the MySQL Site and download/install the ODBC Drivers for Windows here: Click Here

Once you have downloaded the drivers, you can setup your connection and test it. To do this. Go to Start –> Control Panel –> Administrative Tools –> Data Sources (ODBC). This will open the “ODBC Data Source Administrator” dialog box. Click on the “System DSN” tab and click the “Add” button. Locate the entry titled “MySQL ODBC 5.1 Driver” and SELECT it. This will load the MySQL Connector/ODBC Data Source Configuration. Fill out fields in the form and make sure you click the “Test” button to confirm your connection is Successful. Then Click “OK” to Save it. You should now see your connection listed in the System Data Sources. Click “OK” to close the ODBC Data Source Administrator Dialog.

Now that you have your ODBC Connection defined, you are ready to build your Integration Services Package.

BUT, before you begin make sure you follow the instructions in this blog entry: Click Here

This blog entry will give you instructions on how to install an adaptor that you will need to pull data from a list and send it to a database server. Once this adaptor is setup you will be able to add the “SharePoint List Source” tool to your SSIS Package.

One final thing needs to be done in MySQL before you develop the package. When SharePoint sends data via SSIS, there will be quotes associated with text. This will cause a problem when attempting to import the data into MySQL because the database will not know what to do with the quotes around the text data. In other words MySQL is expecting:

HERE IS TEXT                      (Instead of)                    “HERE IS TEXT”

So to fix this problem, you have to go into MySQL and globally configure the database to set “sql_mode” to ANSI_QUOTES. MySQL Describes setting ANSI_QUOTES as:  Treat “"” as an identifier quote character (like the “`” quote character) and not as a string quote character. You can still use “`” to quote identifiers with this mode enabled. With ANSI_QUOTES enabled, you cannot use double quotes to quote literal strings, because it is interpreted as an identifier.

Here is the MySQL reference that describes MySQL SQL MODES: Click Here

With all the above accomplished, you will now be able to build your package.

Some steps to help you build the package:

Add a Data Flow Task to your Control Flow in the package. Go inside the Data Flow Task and add a “SharePoint List Source” control and an “ADO.NET Destination” control. Configure the “SharePoint List Source” control with a SharePoint “SiteListName”, “SiteListViewName”, and “SiteUrl” (Use your All Items view fo rthe “SiteListViewName” for testing).  Check your “Column Mappings” to verify the control is seeing the List Data. Click “OK” to close the control.

Now drag the Green Arrow from the “SharePoint List Source” to the the “ADO NET Destination” to connect the flow. Now configure the ADO NET Destination Control. In the “Connection Manager” section, click the “New…” button. This will open the “Configure ADO.NET Connection Manager”. Click on the “New…” Button inside here.  This will take you to the “Connection Manager” dialog. From the “Provider” drop-down, choose “Odbc Data Provider” fromt the choices under “.Net Providers”. Now, under the “Data Source specification” section (make sure “Use user or system data source name:” is selected) click the drop down and find your previously defined ODBC connection you created above.  Under the “Login information” section, enter the User name and Password that has access to the MySQL Database. Now click the “Test Connection” button to confirm that your connection is successful. It the connection is successfull, click the “OK” button and you will see your MySQL connection listed in the “Configure ADO.NET Connection Manager” dialog. Now click “OK” to get out of this. You should now be back in the “ADO.NET Destination Editor” dialog. In the “Use a table or view” drop down, TYPE the name of a table in your MySQL database (many time the drop down will not populate properly – must be a bug – so you have to type the name of the table). Once you type the name of the table, click the “Preview” button to confirm the connection to the table.  Now click the “Mappings” link on the left side of the Dialog. This will show you what fields you have available from the SharePoint list on the left and what receiving fields you have available in the MySQL database/table on the right. If the fields are matched exactly, SSIS will attempt to predict a match with lines added between the two. If a prediction is not made, just click a field from the left and drag it to a field on the right. It will create a line between the two. Once you are finished mapping the fields, click the “OK” button.

Now you are ready to Run your package. Hopefully, you see “Greenage”. If the package is successful, you’ll see the final step turn green. Check your MySQL Database/table for data.

Now your LAMP developer can develop a reporting system off of the MySQL database.