Understanding Data Sources

This article describes the types of data sources you can create and provides instructions for creating a data source using the Data Source wizard. Before you can create a profile and generate reports, you need to have a configured data source. Configuring a data source tells Webtrends where to find the data it analyzes for reports. Data sources store information about where Webtrends should access your site data. When you specify where your data resides, Webtrends defines the location as a data source, so you can select that location quickly when you create profiles. You must have a configured data source in order to create a profile. If you use Webtrends On Demand or if you use Webtrends software with the SmartSource Data Collector, you also need a configured data source in order to create the JavaScript tracking code used to tag your site and collect data about your web site activity.

You can use any of the following types of data as a data source:

• One or more Web server log files.
• One or more SmartSource Data Collector log files. SmartSource Data Collector creates highly customized logs based on data collected through client-side tagging. For a detailed discussion of client-side tagging, see the Webtrends Guide to Web Analytics.

Types of Data Sources

Webtrends can access your data from different types of data sources.

Web: Log File

Uses Web server log file data as the source for your analysis.

Web: SmartSource Data Files

Uses SmartSource data as the source for your analysis. If you use SmartSource Data Collector to collect your Web data, select this option.

Managed SDC Data Sources

If you only use SDC data sources, Webtrends creates a Managed SDC Account Rollup data source in addition to the SDC data sources you have configured. This data source is not shown in the Data Sources list, but can be selected when you create a profile. The Managed SDC Account Rollup data source combines all the data in your SDC data sources. If you report on multiple data sources, you can select this data source to report on all your data in a single report. The Managed SDC Account Rollup data source does not exist unless you have defined one or more SDC data sources, and you have not defined any non-SDC data sources. If you create a non-SDC data source after assigning the Managed SDC Account Rollup data source to a profile, any profiles that use the Rollup data source will stop working.

Configuring an SDC Data Source

Creating a SmartSource Data Collector (SDC) data source in Webtrends Administration is required before you can generate the Webtrends JavaScript tracking code you need to tag your web site. The tracking code is keyed to a unique ID called a DCSID, short for data collection server id. This ID tells the data collection server what web site or site components to track.

When you create a data source, you also specify the location of your SDC web data files. If you have already defined an SDC data source and you need to know your DCSID, you can edit the data source in Webtrends Administration to find the DCSID.

Creating a Data Source for Webtrends Software

To create an SDC data source for software:

1. In the left pane, click Administration > Application Settings > Data Sources.

2. Click New.

3. In the Data Source Type dialog, select Web: SmartSource Data Files. Click Next.

4. In the Data Location dialog, identify the location of the log files, which are your SmartSource Data Collector data files. Use the location you would identify for standard Webtrends analysis.

5. Provide information about the time zone for the data file. Click Next.

6. In the SmartSource Site ID dialog, click Generate a new SmartSource Site ID and specify the domain name or IP address of the SmartSource Data Collector. Click Next.

7. Read the Summary to ensure your settings are correct, and save the data source.

8. To configure and download JavaScript tracking code associated with this data source, return to the Data Sources list and click Build Tag (Tracking Code). Webtrends Tag Builder opens and is populated with information for the current data source. Tag Builder is a web site that automates JavaScript tracking code creation. You can use the code you create with Tag Builder to tag your web site.

Log File Path Macros

To express relative dates and times, plus (+) and minus (-) modifiers are used within the square brackets of the date macros. For example, "[DD-2]" represents two days ago. Date and time macro modifiers affect the entire macro expression. The following examples illustrate this behavior:

• If today is September 14, 2008. "[YYYY]-[MM]-[DD-3]" will be interpreted as "2008-09-11"
• If today is October 15, 2008. "[YYYY]-[MM-1]-01" will be interpreted as "2008-09-01"
• If today is November 16, 2008. "[YYYY+5]-[MM+3]-[DD]" will be interpreted as "2014-02-16"
• If today is January 1, 2008. "[YYYY]-[MM]-[DD-1]" will be interpreted as "2007-12-31"

Schedule and Date Macro HINTS:

• To get an end-of-month report, schedule a report to run on "1" (first day of month), set the Start Date to "[YYYY]-[MM-1]-01" and set the End Date to "[YYYY]-[MM]-[DD-1]"
• To get an end-of-week (ending on Saturday) report, schedule a report to run on "Sunday", set the Start Date to "[YYYY]-[MM]-[DD-7]" and set the End Date to "[YYYY]-[MM]-[DD-1]"
• To get "daily" reports that cover yesterday's activity, schedule a report to run on "All", set the Start Date to "[YYYY]-[MM]-[DD-1]" and set the End Date to "[YYYY]-[MM]-[DD-1]"

Using Webtrends Analytics software, you can use the following macros to specify log file paths that retrieve files for specific dates:

%date-n%
Subtracts n days from current date. For example, if today is February 4, 2008 the macro %date-5%%mm%%dd%%yyyy% produces 01302008. This macro must precede any other macros.

%date+n%
Adds n days to current date. For example, if today is February 4, 2008, the macro %date+5%%mm%%dd%%yyyy% produces 02092008. This macro must precede any other macros.

%dd%
Replaces macro with current day of the month. Value is expressed as a number, for example 02.

%dd-n%
Subtracts n days from the current day of the month and replaces the macro with a value expressed as a number. For example, if the current day is the 20th of the month, the macro %dd-8% produces a value of 12.

%dd+n%
Adds n days to the current day of the month and replaces the macro with a value expressed as a number. For example, if the current day is 10th of the month, the macro %dd+7% produces a value of 17.

%dw%
Replaces macro with current day of week. Value is expressed as a number within the range of 0 - 6, with Sunday being 0 and Saturday being 6.

%dy%
Replaces macro with current day of year. Value is expressed as a number with range of 1 - 366.

%hour%
Replaces macro with current hour.

%minute%
Replaces macro with current minute.

%mm%
Replaces macro with current month. Value is expressed as a number, for example 01.

%mm-n%
Subtracts n months from the current month and replaces the macro with a value expressed as a number. For example, if the current month is May (05), the macro %mm-3% produces 02 (February).

%mm+n%
Adds n months to the current month and replaces the macro with a value expressed as a number. For example, if the current month is May (05), the macro %mm+3% produces 08 (August).

%Mon%Replaces
macro with current month. The first letter of the month is capitalized and the value is expressed as string, for example Jan.

%mon%
Replaces macro with current month. The month is in all lower case and the value is expressed as a string, for example jan.

%MON%
Replaces macro with current month. The month is in all upper case and the value is expressed as string, for example, JAN.

%month%
Same as %mm% (see above).

%second%
Replaces macro with current second.

%wm%
Replaces macro with current week of the month. Value is expressed as a number with range of 0 - 5, for example, 4. This value is not affected by date- or date+ macros.

%wy%
Replaces macro with current week of year. Value is expressed as a number with range of 0 - 53.%yday%Replaces macro with date of yesterday. Value is expressed as a number, for example 25. This value is not affected by date- or date+ macros.

%ymonth%
Replaces macro with yesterday’s month. Value is expressed as a number with range of 1 - 12, for example 6. Is not affected by date- or date+ macros.

%yy%
Replaces macro with current year. Value is expressed as a two-digit number, for example, 02.

%yy-n%
Subtracts n days from the current year and replaces the macro with a value expressed as a two-digit number. For example, if the current year is 2008, the macro %yy-1% produces a value of 07.

%yy+n%
Adds n days to the current year and replaces the macro with a value expressed as a two-digit number. For example, if the current year is 2008, the macro %yy+4% produces a value of 12.

%yyear%
Replaces macro with yesterday’s year. Value is expressed as two digit number, for example 01. Is not affected by date- or date+ macros.

%yyyy%
Replaces macro with current year. Value is expressed as a four digit number, for example 2008.

%yyyy-n%
Subtracts n years from the current year and replaces the macro with a value expressed as a four-digit number. For example, if the current year is 2008, the macro %yyyy-1% produces a value of 2007.

%yyyy+n%
Adds n years to the current year and replaces the macro with a value that’s expressed as a four-digit number. For example, if the current year is 2008, the macro %yyyy+4% produces a value of 2012.

Configuring a Web Server-Based Data Source

You can use the web server log file data as the source for your analysis of web server-based data source.

To create a web server-based data source:

1. In the left pane, click Administration > Application Settings > Data Sources.

2. Click New.

3. In the Data Source Type dialog, select Web: Log File and click Next.

4. Read the warning message about how creating a non-SDC data source will cause any existing roll up profiles to quit working and click Next. For more information about roll-up profiles, see “Managed SDC Data Sources,” above.

5. Identify the location of the log files, which are your web server-based data. If your log files are located on a computer other than the analysis computer, you may need to use FTP to move them to the analysis computer or to a shared drive. However, accessing log files through an FTP server slows down Webtrends analysis, especially if several profiles use the same data source and Webtrends must download the same file multiple times.

6. Save the data source.