The Splunk App for Web Analytics currently supports data from Apache and IIS logs. Make sure you use the sourcetype access_common, access_combined or iis for this data. If you already have data in Splunk under a different sourcetype you can use sourcetype renaming or by modifying the eventtype web-traffic to include the names of your sourcetypes.
The app comes with two sets of sample data for Apache and IIS. You can enable these static sample inputs by going into Settings->Data inputs->Files & Directories
If your data is stored in an index that is not searched by default for your Splunk user, you need to add All non-internal indexes (or the specific index in question) to the Selected indexes in Access controls -> Roles -> [ROLE NAME]
The Splunk App for Web Analytics works in a multi website environment. Websites are configured from a combination of the host and the source field. Each event with that unique combination will be tagged with the corresponding website name in the field "site". You can use wildcards (*) in the Source and Host field to select multiple files matching a pattern. There is a website setup form page that allows you to add these in an easy way.
Here are some examples of valid website configurations with or without wildcards
No wildcards
Site Host Source
roadrunner.com server1 /var/log/httpd/access_log
roadrunner.com server2 /var/log/httpd/access_log
With wildcards
Site Host Source
roadrunner.com server /var/log/httpd/access_
The data in the setup form will be stored in the lookup file called WA_settings.csv. You can also manually edit this file. The websites setup page can be found under Setup->Websites.
Once the data has been imported run the two lookups "Generate user sessions" and "Generate pages". These will be used throughout the app. Once run the first time, they will automatically be updated via two scheduled searches that runs every 10 minutes that adds any new data coming into the app. Running these lookup searches might take a long time depending on how much data you have in Splunk but its important you let the searches finish before you move on to the next step. If you have too much data to run these for everything you can modify the time period to something less than "All time" which is the default time period. The lookup reports can be found under Setup->Lookups or by using the links above. It's important that thes searches return results. If not, the app will not work.
The Splunk App for Web Analytics uses data model acceleration extensively to power the dashboards. Once the lookups in the previous step has completed you should enable acceleration for the data model "Web". The data model can be found under Settings->Data models. Set the summary range appropriately depending on how long you want to keep the data, > 1 Month. The data model is updated every 10 minutes in order for the sessions to get picked up properly. The data model acceleration needs to finish before you will see any data in any dashboard except the "Real-Time" dashboard which uses raw log data as source. That means that you initially might not see data until the data model has finished building. This could initially take up to an hour depending on how much data it is trying to build.
If you want to monitor certain browsing paths or pageviews you can configure goals. This is used if you for instance want to get conversion rates or funnel abandonment rates. You can find the Goals setup page under Goals->Goals Setup.
The goals are stored in a summary index called "goal_summary".
When enabling goals, the app will start monitor goal completions from the time you save the goal. To backfill goals there is a search called "Generate Goal summary - Backfill" which can be found under the Goals menu. Please note that running this search multiple times will mean the goal completions will be duplicated. To reset the goals you need to clean the "goal_summary" index.
In the context of the app, try and do the search for:
tag=web
If this is not returning any results I suspect you are not seeing the data because it is stored in a non-default index and the user in Splunk does not search in non-default indexes automatically. Another issue might be that you are not using any of the pre-configured sourcetypes. See Setup point 1 above.
If this is returning results, double check that each entry has the "site" field populated. It's crucial that this field exists in your data. See Setup point 2 above.
As the app relies heavily on data model accelerations you will not see anything in any dashboards (except the "Real-Time" ones) until this acceleration has completed. Initially this could take a while. There is a "Data Model Audit" dashboard that will tell you if the acceleration is complete or not.
The user agent parsing is based on an add-on developed by David Shpritz (TA-user-agents) who in turn uses a Python module from:
https://github.com/tobie/ua-parser
Please note that this upgrade will require a data model rebuild. See documentation for upgrade instructions.
Minor release
- Performance tweaks on the dashboards to use post-process searches were possible
- Refactored the scheduled searches to make it easier during initial install
- Changed all knowledge objects of the app to be visible to the app only instead of globally
- Updated the eventtypes
- Changed session cutoff time to 30 minutes instead of 15 minutes
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.