Friday, March 4, 2011

Google Custom Search in C#

This post contains a C# class to search the internet using Google’s custom search API. Before using this class you must first create a Google account (link) and generate an API key (link).

Unfortunately, this code cannot be used directly from a Silverlight application because of security limitations (see cross domain policy). If you need this capability in a Silverlight web application I would suggest performing the search via an ASP.NET proxy (license permitting).

In this sample, I called the Google search API from a WPF application, but it would also work in a WinForms or ASP.NET application as suggested above.

Here is the XAML of the Window that displays the results of the Google search.

<Window x:Class="ESRI.PrototypeLab.MapServiceHarvester.MainWindow"
       xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
       xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
       Height="600"
       Width="800"
       >
    <Grid>
<DataGrid x:Name="DataGridResults" AutoGenerateColumns="True" />
    </Grid>
</Window>

Here is the code behind that makes that performs the internet search and displays the results in the datagrid defined above.

public partial class MainWindow : Window {
    public MainWindow() {
        InitializeComponent();

        this.Loaded += (s, e) => {
            GoogleSearch search = new GoogleSearch() {
                Key = "<enter your key here>",
                CX = "013036536707430787589:_pqjad5hr1a"
            };
            search.SearchCompleted += (a, b) => {
                this.DataGridResults.ItemsSource = b.Response.Items;
            };
            search.Search("gis");
        };
    }
}

And here is the class that calls Google’s custom search API, desterilizes the JSON response and returns the result as single .NET objects.

public class GoogleSearch {
    public GoogleSearch() {
        this.Num = 10;
        this.Start = 1;
        this.SafeLevel = SafeLevel.off;
    }
    //
    // PROPERTIES
    //
    public string Key { get; set; }
    public string CX { get; set; }
    public int Num { get; set; }
    public int Start { get; set; }
    public SafeLevel SafeLevel { get; set; }
    //
    // EVENTS
    //
    public event EventHandler<SearchEventArgs> SearchCompleted;
    //
    // METHODs
    //
    protected void OnSearchCompleted(SearchEventArgs e) {
        if (this.SearchCompleted != null) {
            this.SearchCompleted(this, e);
        }
    }
    public void Search(string search) {
        // Check Parameters
        if (string.IsNullOrWhiteSpace(this.Key)) {
            throw new Exception("Google Search 'Key' cannot be null");
        }
        if (string.IsNullOrWhiteSpace(this.CX)) {
            throw new Exception("Google Search 'CX' cannot be null");
        }
        if (string.IsNullOrWhiteSpace(search)) {
            throw new ArgumentNullException("search");
        }
        if (this.Num < 0 || this.Num > 10) {
            throw new ArgumentNullException("Num must be between 1 and 10");
        }
        if (this.Start < 1 || this.Start > 100) {
            throw new ArgumentNullException("Start must be between 1 and 100");
        }
                            
        // Build Query
        string query = string.Empty;
        query += string.Format("q={0}", search);
        query += string.Format("&key={0}", this.Key);
        query += string.Format("&cx={0}", this.CX);
        query += string.Format("&safe={0}", this.SafeLevel.ToString());
        query += string.Format("&alt={0}", "json");
        query += string.Format("&num={0}", this.Num);
        query += string.Format("&start={0}", this.Start);
           
        // Construct URL
        UriBuilder builder = new UriBuilder() {
            Scheme = Uri.UriSchemeHttps,
            Host = "www.googleapis.com",
            Path = "customsearch/v1",
            Query = query
        };

        // Submit Request
        WebClient w = new WebClient();
        w.DownloadStringCompleted += (a, b) => {
            // Check for errors
            if (b == null) { return; }
            if (b.Error != null) { return; }
            if (string.IsNullOrWhiteSpace(b.Result)) { return; }

            // Desearealize from JSON to .NET objects
            Byte[] bytes = Encoding.Unicode.GetBytes(b.Result);
            MemoryStream memoryStream = new MemoryStream(bytes);
            DataContractJsonSerializer dataContractJsonSerializer =
new DataContractJsonSerializer(typeof(GoogleSearchResponse));
            GoogleSearchResponse googleSearchResponse =
dataContractJsonSerializer.ReadObject(memoryStream) as GoogleSearchResponse;
            memoryStream.Close();

            // Raise Event
            this.OnSearchCompleted(
                new SearchEventArgs() {
                    Response = googleSearchResponse
                }
            );
        };
        w.DownloadStringAsync(builder.Uri);

    }
}

public enum SafeLevel { off, medium, high }

public class SearchEventArgs : EventArgs {
    public GoogleSearchResponse Response { get; set; }
}

[DataContract]
public class GoogleSearchResponse {
    [DataMember(Name = "kind")]
    public string Kind { get; set; }
    [DataMember(Name = "url")]
    public Url Url { get; set; }
    [DataMember(Name = "queries")]
    public Queries Queries { get; set; }
    [DataMember(Name = "context")]
    public Context Context { get; set; }
    [DataMember(Name = "items")]
    public List<Item> Items { get; set; }
}

[DataContract]
public class Url {
    [DataMember(Name = "type")]
    public string Type { get; set; }
    [DataMember(Name = "template")]
    public string Template { get; set; }
}

[DataContract]
public class Queries {
    [DataMember(Name = "nextPage")]
    public List<Page> NextPage { get; set; }
    [DataMember(Name = "request")]
    public List<Page> Request { get; set; }
}

[DataContract]
public class Page {
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "totalResults")]
    public int Request { get; set; }
    [DataMember(Name = "searchTerms")]
    public string SearchTerms { get; set; }
    [DataMember(Name = "count")]
    public int Count { get; set; }
    [DataMember(Name = "startIndex")]
    public int StartIndex { get; set; }
    [DataMember(Name = "inputEncoding")]
    public string InputEncoding { get; set; }
    [DataMember(Name = "outputEncoding")]
    public string OutputEncoding { get; set; }
    [DataMember(Name = "safe")]
    public string Safe { get; set; }
    [DataMember(Name = "cx")]
    public string CX { get; set; }
}

[DataContract]
public class Context {
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "facets")]
    public List<List<Facet>> Facets { get; set; }
}

[DataContract]
public class Facet {
    [DataMember(Name = "label")]
    public string Label { get; set; }
    [DataMember(Name = "anchor")]
    public string Anchor { get; set; }
}

[DataContract]
public class Item {
    [DataMember(Name = "kind")]
    public string Kind { get; set; }
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "htmlTitle")]
    public string HtmlTitle { get; set; }
    [DataMember(Name = "link")]
    public string Link { get; set; }
    [DataMember(Name = "displayLink")]
    public string DisplayLink { get; set; }
    [DataMember(Name = "snippet")]
    public string Snippet { get; set; }
    [DataMember(Name = "htmlSnippet")]
    public string HtmlSnippet { get; set; }
    [DataMember(Name = "cacheId")]
    public string CacheId { get; set; }
    //[DataMember(Name = "pagemap")] *** Cannot deserialize JSON to .NET! ***
    //public Pagemap Pagemap { get; set; }
}

[DataContract]
public class Pagemap {
    [DataMember(Name = "metatags")]
    public List<Dictionary<string, string>> Metatags { get; set; }
}

[DataContract]
public class Metatag {
    [DataMember(Name = "creationdate")]
    public string Creationdate { get; set; }
    [DataMember(Name = "moddate")]
    public string Moddate { get; set; }
}

And lastly, here is the result.

image

The only issue I had with this sample was trying to deserialize the JSON associated with the “pagemap” property (see commented out code above). Any tips would be appreciated.

8 comments:

  1. Hi, Thanks For code,

    But its give error 400 bad request.

    ReplyDelete
  2. Did you generate your own key?

    ReplyDelete
  3. Is it possible to use this code from a console application ? I am trying to make a google search and just take the first URL returned by google. Any suggestions would be appreciated.

    ReplyDelete
  4. Sure!
    Use this code:

    GoogleSearch search = new GoogleSearch() {
    Key = "",
    CX = "013036536707430787589:_pqjad5hr1a"
    };
    search.SearchCompleted += (a, b) => // here goes your your method to write the results to the console;
    search.Search("gis");

    ReplyDelete
  5. Why you don't attach a complete solution ?

    ReplyDelete
  6. Hello, your post help me a lot.
    But, i need to make my searh in google.es domain, where i can define the search domain to use in my queryes?-
    Best regards

    ReplyDelete
  7. Other question, how can i obtain the searh result, in the same order thats browser show?.
    Example:
    Make a query in www.google.es browser, the result is:
    1.- Site 1
    2.- Site 2
    3.- Site 3

    Using this code, the resilt is
    1.- Site 3
    2.- Site 1
    3.- Site 2

    How can i obtain the same result that query browser?

    ReplyDelete
  8. This was a great help! One thing I ran into was that certain characters, such as #, and the query wouldn't go through at all. Using System.Web.HttpUtility.UrlEncode fixed that for me.

    ReplyDelete