Most Popular/Visited Articles in Sitecore Part-2

As we already discussed about API to get Most Popular/Visited Pages in last post. Here we will discuss how to manage the data for most visited pages by scheduler and calling the above API only one time in a day. If you haven’t read my last post, please click here to read.

At the beginning we decided to go with Sitecore scheduler to execute the API for same but after digging into some posts I found that if you want to run the Scheduler on specific time then Sitecore scheduler is not the recommended way to go with. There are many posts which suggest doing customization in current code and make the scheduler to work on specific time but still I found it inconsistent. So finally, we thought to go with windows scheduler on CM server.

We have created 3 below fields for same –

  1. Include templates for Most Popular Pages – Treelist field to set the data templates for which types of pages we want to get.
  2. Most Popular Items – TreelistEx field to store items after calculating by API.
  3. Number of Results – Number field to set number of results to show on page.

Below is the API we have used to execute by scheduler and store the most visited pages –

try
            {
                //  Calling the API which is given in previous post to get the most popular pages
                var mostVisitedPages = MostPopularPageEvents.GetMostVisitedPages();
                if (mostVisitedPages.Any())
                {
                    Sitecore.Data.Database master = Sitecore.Data.Database.GetDatabase(“master”);
                    if (master != null)
                    {
                        var siteItem = master.GetItem(“Site Item ID”);
                        if (siteItem!= null)
                        {
                            int pageCounts = 5;
                            List<Item> itemsToSaveInMostPopular = new List<Item>();
                            var mostVisitedItems = mostVisitedPages.Select(i => master.GetItem(new Guid(i).ToString())).Where(i => i != null);
                            if (siteItem.FieldHasValue(Templates.MostPopularPages.Fields.NoOfItems))
                            {
                                pageCounts = int.Parse(siteItem [Templates.MostPopularPages.Fields.NoOfItems]);
                            }

                            List<Item> templatesTypes = null;
                            if (siteItem.FieldHasValue(Templates.MostPopularPages.Fields.IncludeTemplates))
                            {
                                templatesTypes = siteItem.GetMultiListValueItems(Templates.MostPopularPages.Fields.IncludeTemplates).ToList();
                            }

                            foreach (var itm in templatesTypes)
                            {
                                if (itm != null)
                                {
                                    var templateSpecificItems = mostVisitedItems.Where(i => i.TemplateID.Equals(itm.ID)) ToList();
                                    if (templateSpecificItems.Any())
                                    {
                                        itemsToSaveInMostPopular.AddRange(templateSpecificItems.Take(pageCounts));
                                    }

                                }
                            }

                            using (new Sitecore.SecurityModel.SecurityDisabler())
                            {
                                using (new EditContext(siteItem))
                                {
                                    siteItem[Templates.MostPopularPages.Fields.MostPopularItems] = string.Join("|", itemsToSaveInMostPopular.Select(i => i.ID.ToString()));
                                }
                            }

                            LogManager.Diagnostic("Most Popular " + string.Join("|", itemsToSaveInMostPopular.Select(i => i.ID.ToString())), true);
                        }
                    }
                }

            }
            catch (Exception ex)
            {
                LogManager.Error("Error with Most Popular Scheduler" + ex.Message, true);
            }

With the help of above API we are able to get the most popular pages for specific template types also with specific count and storing them into Most Popular Items field of Sitecore. After the execution of this API we are publishing the item programmatically at the same time if everything works perfect.

Here we also made some updates if there is no data coming from the API (Because of any exceptions) then we are not updating the field value it use old data to display.

We have created windows scheduler to run the API everyday and the above API …. getting most popular pages and filtering the results specifically for the templates which we needed. And then storing only specific number of items which we need to show on website instead of storing large number of records.

There may be many other ways to handle the same but I am sharing what I did for my implementation.

Advertisements
Posted in Reporting, Sitecore, Sitecore Analytics, Sitecore Reporting | Tagged , | Leave a comment

Most Popular/Visited Articles in Sitecore

I am writing post after such a long time… and this time I found something interesting and useful which may save your development time.

In my recent project we got requirement to show Most Popular/Visited articles on website. Sitecore provided great analytics solutions where it stores many reports including page views/Visits.

I was already aware that there is nothing to do with reporting side as Sitecore out of the box provided features and stores data in reporting db. I just need to look for the API to get the correct data for Most Popular/Visited pages.

I started searching on internet and found some interesting posts where we need to call SQL query to get the Most Visited Pages results. After some research I found an API from my last project which was quite similar and implemented by one of my colleague @Julia. It was for to get Top Searched Keywords from analytics reports.

I copied the code from there to my solution and made some changes as per my requirement to get correct data from reporting DB.

Below is the API which I used to get the Most Popular pages in Sitecore –


        public static IEnumerable<string> GetMostVisitedPages()
        {
            try
            {
                var reportingService = ApiContainer.Repositories.GetReportingService();
                var reportQuery = GetReportQuery();
                ReportResponse reportResponse = reportingService.RunQuery(reportQuery);
                if (reportResponse != null && reportResponse.Data != null && reportResponse.Data.Localization != null && reportResponse.Data.Localization.Fields != null && reportResponse.Data.Localization.Fields.Any())
                {
                    var searchFields = reportResponse.Data.Localization.Fields.FirstOrDefault();
                    if (searchFields != null)
                    {
                        return searchFields.Translations.Select(r => r.Key).ToList();
                    }
                }
            }
            catch (Exception ex)
            {
                Log.Error(ex.Message, ex, typeof(MostPopularPageEvents));
            }
            return new List<string>();
        }

        private static ReportQuery GetReportQuery()
        {
            ReportQuery reportQuery = new ReportQuery();
            reportQuery.Site = "SUM";
            reportQuery.Segments = new string[] { ID.Parse("{7B339703-2C03-4C19-89C9-53930476A59F}").ToShortID().ToString() };
            reportQuery.Keys = new string[] { "ALL" };
            reportQuery.Fields = null;
            reportQuery.Parameters.DateFrom = DateTime.Now.AddMonths(-6);
            reportQuery.Parameters.DateTo = DateTime.Now.AddDays(1);
            reportQuery.Parameters.TimeResolution = TimeResolution.Collapsed;
            reportQuery.Parameters.KeyTop = 10000000;
            reportQuery.Parameters.KeySkip = 0;
            reportQuery.Parameters.PadEmptyDates = true;
            reportQuery.Parameters.KeyOrderBy = new FieldSort() { Direction = SortDirection.Desc, Field = SortField.PageViews, FieldName = "Visits" };
            reportQuery.Parameters.KeyFromParent = string.Empty;
            reportQuery.Parameters.KeyFromAncestor = string.Empty;
            var propertyInfo = reportQuery.GetType().GetProperty("RequestType", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
            propertyInfo.SetValue(reportQuery, System.Convert.ChangeType(Enum.Parse(propertyInfo.PropertyType, "0"), propertyInfo.PropertyType), null);
            return reportQuery;
        }

In the given API item id ({7B339703-2C03-4C19-89C9-53930476A59F}) is the id of All visits by page item from /sitecore/system/Marketing Control Panel/Experience Analytics/Dimensions/Pages/By page/All visits by page location.

As per my understanding here we don’t need to make any changes in API we can directly use the given code for implementation. Also, if we want to get other reports we just need to make small tweaks in API and we can achieve that.

There might be other methods also to implement the same with Xconnect but as I have already confirmed from Sitecore and they recommended they way we are doing it.

Story is still not ends here now the task is we can’t call the given API all the time on page load as it is not good for page load performance to get the millions of records/interactions and process them to get necessary data. So, we have implemented it by setting up a scheduler. We will discuss the same in next post.

Thanks @Julia Gavrilova for helping me.

Posted in Reporting, Sitecore, Sitecore Analytics, Sitecore Reporting | Tagged , , , | 1 Comment

Enable SSL for sending emails on Sitecore 9

In my current implementation I was asked to use SMTP settings which only supports connections over TLS or SSL. I am using Sitecore 9 in my current project.

By default, Sitecore does not use SSL for sending emails, and this is something that you need to adjust yourself by adding enableSSL in your web.config file.

This configuration only effects the emails that Sitecore send for there internal use like forgot Sitecore password and also If you are using Sitecore OOTB or default function for sending emails for example Sitecore.MainUtil.SendEmail method.

In previous versions, we configured the SSL-setting in a standard ASP.NET fashion, by adding the below setting into web.config file:

<system.net>
 <mailSettings>
    <smtp deliveryMethod="Network">
     <network enableSsl="true" />
   </smtp>
  </mailSettings>
</system.net>

But in Sitecore 9 it doesn’t work anymore. I didn’t find any post about that there is some changes in configration for not to use settings given in web.config file. I thought there is some issue with my set up so made many changes in that. After almost wasting 4-5 hours and trying many things I started looking into different direction.

Now I was pretty sure that there is some changes on Sitecore side. So I started digging into the dll’s. And finally after digging into Sitecore.Kernel I found that Sitecore has introduced a new setting for this, which I am not sure in which version of Sitecore this was changed, but I do know that this is with Sitecore9.x.

Adding <setting name=”MailServerUseSsl” value=”true”/> will work in Sitecore 9 for SSL

Below is the new configuration settings for Sitecore9 – 

<sitecore>     
       <settings>
           <setting name="MailServer" value="[enter value]"/>
           <setting name="MailServerUserName" value="[enter value]"/>
            <setting name="MailServerPassword" value="[enter value]"/>
            <setting name="MailServerPort" value="[enter value]"/>
	     <setting name="MailServerUseSsl" value="true"/>
        </settings>
    </sitecore>

I am sure if you are going to start with Sitecore 9 and planing to use Sitecore forms for sending emails or simple .net forms and specially if your smtp settings are working over SSL then it can save your some time if you are not aware about this change.

Posted in Sitecore, Uncategorized | Tagged , , | Leave a comment

Custom Alias Resolver for Sitecore media item

Working on different projects and different type of tasks is always give an opportunity to learn something new as with my last implementation we came across with a very different requirement where client ask us to make the media url’s more friendly as Sitecore content tree URL’s. Or in other words if I say they ask us to build something which look like a Content Page but render like a media item. And that media can be sections related (Means URL for that media can be manageable as content tree architecture) So here I am going to write what I have build so far to complete this task.

We have created a Data Template (Media Reference) type with Droptree field. Droptree field we provided to map the media item. My plan was to give an option to create a simple content item with Media Reference template type and map the media item into droptree field.

It was easy to create but now the task was render the mapped media item onto media reference page. Here what I already know there is no out of the box functionality available to achieve this I need to customize any of the Sitecore pipeline which actually run the Item URL but render the mapped media item.

After reading few articles I got the understanding that If I can create Custom Alias Resolver than I can achieve this and I started digging into Sitecore dll’s for Alias Resolver.

I have created a class CustomAliasResolver and inherited by HttpRequestProcessor.

After that I copied the code of Alias Resolver into this class and made the tweaks as per my need. See below code to achieve this –


public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull((object)args, "args");
            Database database = Context.Database;
            if (database == null)
            {
                Tracer.Warning((object)"There is no context database.");
            }
            else
            {
                Profiler.StartOperation("Resolve custom alias.");
                if (ProcessItem(args))
                {
                    if (Context.Item != null)
                    {
                        string mediaUrl = MediaManager.GetMediaUrl((MediaItem)Context.Item);
                        if (!string.IsNullOrEmpty(mediaUrl))
                        {
                            string handler = HandlerUtil.GetHandler(mediaUrl);
                            if (!string.IsNullOrEmpty(handler))
                            {
                                Context.Data.RawUrl = mediaUrl;
                                args.Context.RewritePath(handler, mediaUrl, args.Url.QueryString, true);
                                args.AbortPipeline();
                            }
                            Context.Page.FilePath = mediaUrl;
                        }
                    }
                }
                Profiler.EndOperation();
            }
        }


        private static bool IsMediaItem(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            return item.Paths.IsMediaItem && item.TemplateID != TemplateIDs.MediaFolder;
        }

        private static string GetAbsoluteMediaUrl(MediaItem mediaItem)
        {
            string relativeUrl = MediaManager.GetMediaUrl(mediaItem);
            return WebUtil.GetFullUrl(relativeUrl);
        }


        private bool ProcessExternalUrl(HttpRequestArgs args)
        {
            string targetUrl = Context.Database.Aliases.GetTargetUrl(args.LocalPath);
            if (targetUrl.Length > 0)
                return this.ProcessExternalUrl(targetUrl);
            return false;
        }


        private bool ProcessExternalUrl(string path)
        {
            if (Context.Page.FilePath.Length > 0)
                return false;
            Context.Page.FilePath = path;
            return true;
        }


        private bool ProcessItem(HttpRequestArgs args)
        {
            var item = Sitecore.Context.Database.GetItem(args.Url.ItemPath);
            if (item != null && item.TemplateID == Templates.MediaReference.ID)
            {
                if (!string.IsNullOrEmpty(item[Templates.MediaReference.Fields.ContentReference]))
                {
                    ID targetId = new ID(item[Templates.MediaReference.Fields.ContentReference]);
                    if (!targetId.IsNull)
                    {
                        Item target = args.GetItem(targetId);
                        if (target != null)
                            this.ProcessItem(args, target);
                        return true;
                    }
                }

            }
            Tracer.Error((object)("An alias for \"" + args.LocalPath + "\" exists, but points to a non-existing item."));
            return false;
        }

       
        private void ProcessItem(HttpRequestArgs args, Item target)
        {
            if (Context.Item != null)
                return;
            Context.Item = target;
            Tracer.Info((object)("Using alias for \"" + args.LocalPath + "\" which points to \"" + (object)target.ID + "\""));
        }

After that I created a .config file and added below pipeline code into that config file. In below code added new processor to handle the custom Alias Resolver requests. And I added it just before  “AliasResolver” processor.

<sitecore>
  <pipelines>
    <httpRequestBegin>
        <processor patch:before="processor[@type='Sitecore.Pipelines.HttpRequest.AliasResolver, Sitecore.Kernel']" type="SitecoreDemo.Feature.MediaReference.CustomAliasResolver.CustomAliasResolver, SitecoreDemo.Feature.MediaReference"/>
    </httpRequestBegin>
   </pipelines>
 </sitecore>

After the above configuration and code I was able to render Media Item over the Sitecore Item. This code was for media item but we can achieve multiple similar types of requirement with the same approach.

Posted in Sitecore | Tagged , , | Leave a comment

Authenticate External System user’s into Sitecore as Virtual user

In my recent project we got a requirement where need to manage client current site user’s into Sitecore. First we thought to get User DB from client side and configure that with our current solution and use them in current implementation. But as far as client is having there own team to manage there UMS (user management system) which is build in java also they are regularly updating user details in there system. And it was a huge data of user ‘s which was very difficult to manage in Sitecore.

At the final staged we agreed to get the data or user details with the help of web services.

For example if we need to validate user for login than we need to provide the username and password of user and web service was resposible to validate the user as well as provide the required details of user for example  – Name, email id, phone etc.

Now here we started thinking to manage that web service user as Sitecore extranet user so we don’t need to worry about the security permission with Sitecore. Because many of the items in content tree was covered under Role based security.

First we started thinking to create a user in Sitecore at the time of login and then consider the user as extranet users and at the time of log out remove that user from the Sitecore. But after some search on Google I found Virtual user concept and it was really nice to manage the same thing which we were looking for because we were also looking for the virtual set up of user not completely into Sitecore. But after creating virtual user it work as a Sitecore extranet user.

Here we move with our study on virtual user  –

At the time of login user is providing username and password and click on sign in button first we are calling the web service that is basically validating the user from client UMS and than sending the user details in response.

Once we are getting the valid user response then we created virtual user and assign the values of user properties, below is the sample API to create virtual user in Sitecore –

var virtualUser = Sitecore.Security.Authentication.AuthenticationManager.BuildVirtualUser(userName, true);

here username is domain/username.

We also need to maintains multiple custom profile properties so we have created custom user profile item as given in this post.

Below is the sample API for assigning the custom values to custom user profile.

 virtualUser.Profile.FullName = webserviceResponse.response.firstName ?? String.Empty + webserviceResponse.response.lastName ?? String.Empty;
 virtualUser.Profile.Email = webserviceResponse.response.email ?? String.Empty;
 virtualUser.Roles.Add(Sitecore.Security.Accounts.Role.FromName("domain\\RoleName"));  
 virtualUser.Profile.SetCustomProperty(Constants.UserCustomProfileCons.FirstName, webserviceResponse.response.firstName ?? String.Empty);
 virtualUser.Profile.SetCustomProperty(Constants.UserCustomProfileCons.LastName, webserviceResponse.response.lastName ?? String.Empty);
 virtualUser.Profile.SetCustomProperty(Constants.UserCustomProfileCons.profileUID, webserviceResponse.response.profileUID ?? String.Empty);
 virtualUser.Profile.Save();

After storing all the custom profile values with virtual user we logged in virtual user by using Sitecore Authentication code as given below –

AuthenticationManager.LoginVirtualUser(virtualUser)

Now this work as same as Sitecore extranet user login. If we want to check if user is authenticated or not we can use the Sitecore API for same as below –

Sitecore.Context.User.IsAuthenticated;
This API with return true if you are logged in with Virtual user

Below are some examples to get user details on other pages after login with virtual user –

//To Get the username of current logged in user
Sitecore.Context.User.Name;

// TO get the value of any of the custom property
Sitecore.Context.User.Profile.GetCustomProperty("CustomPropertyName");

We have tested it with multiple login at the same time and it is working perfect for us. Please write your feedback in comment section if you find any issue with this approach or if we can make it more better.

 

Posted in Sitecore, Sitecore Authentication, Sitecore Security | Tagged , , , | Leave a comment

Custom Collection of Items in Sitecore Multilist field datasource

I recently got a requirement to show upcoming News (News which date is greater then today’s date) into multilist field on Sitecore Item from News collections in content tree. So after looking into the requirement first thing came in my mind to create a custom Multilist field as we need to show dynamic collections of news on the bases  of current date (Today’s date). Or we need to provide custom datasource to multilist which should be driven by our custom code.

As much as I know about datasource is we can use fast query and query in that but I was not sure we can use custom code also in Datasource field. And I am sure many of us don’t know about the same.

I can see few of blog post about the same but I think because of keyword or meta data it is very difficult to find so I thought to write it again.

First I have created a class file and inherited that class with IDataSource interface (We need to provide reference of Sitecore.Buckets.dll for IDataSource).  And then I have implemented ListQuery method which is defined in IDataSource interface.

ListQuery method return type is Array of items so here we can do our custom code to get the array of items and return that collection as given in below script  –

In below code I am returning upcoming news items  –

public Item[] ListQuery(Item item)
        {
            item = Sitecore.Context.ContentDatabase.GetItem("/sitecore/content/Habitat/Home/Modules/Feature/News/News");
            return item.Axes.GetDescendants().Where(i=>i.IsDerived(Templates.NewsArticle.ID)).Where(i=>DateUtil.IsoDateToDateTime(i[Templates.NewsArticle.Fields.Date])>DateTime.Now).ToArray();
        }
        

And then I logged in to Sitecore and defined the Datasource in datasource field. Now in datasource field we need to define  code:NameSpace.ClassName, AssmeblyName as per given in below screenshot.

We will come with something new in next post…..

Posted in Sitecore, Sitecore Multilist, Sitecore Search | Tagged , , , | Leave a comment

Track search Keywords on Sitecore website

While working on Sitecore search we got a requirement where we need to store keyword s which is visitor will search in our website. First I thought to store them into Sitecore Database by creating items. But once I started looking into analytics feature I got that Sitecore analytics provides feature to store Site searches (Keywords). And also it provide more feature after storing them.

So I planned to use the analytics to store our site search keywords into analytics.

I created below function to store the keyword into sitecore analytics database  –


public bool IsActive => Tracker.Current != null && Tracker.Current.IsActive;

 public virtual void TrackSiteSearch(Item pageEventItem, string query)
        {
            Assert.ArgumentNotNull(pageEventItem, nameof(pageEventItem));
            Assert.IsNotNull(pageEventItem, $"Cannot find page event: {pageEventItem}");
            if (this.IsActive)
            {
                var pageEventData = new PageEventData("Search", [Search Event Item ID])
                {
                    ItemId = pageEventItem.ID.ToGuid(),
                    Data = query,
                    DataKey = query,
                    Text = query
                };
                var interaction = Tracker.Current.Session.Interaction;
                if (interaction != null)
                {
                    interaction.CurrentPage.Register(pageEventData);
                }
            }
        }

[Search Event Item ID] – It is the item id of the search event item created on /sitecore/system/Settings/Analytics/Page Events/Search location of Sitecore.

After creating this function you just need to call the function after getting the search results as below –

  if (!string.IsNullOrWhiteSpace(query) && results.Any())
     {
          TrackSiteSearch(ContextItem.Item, query.QueryText);
     }

This will store keyword into analytics database whenever visitor will perform search.

Where we can see the stored keywords in Sitecore analytics reports –

1. Open the ExperienceAnalytics dashboard from Launch pad as given in below screenshot

launch pad.png

2. Navigate to Behavior section and click on Internal search it will show complete report of searched keywords with all other analytics details as shown below.

 

launch pad2

 

In next post we will discuss how to show top searches keywords in site.

 

Posted in Sitecore, Sitecore Analytics | Tagged , | Leave a comment