Friday, March 4, 2011

Google Custom Search in C#

This post contains a C# class to search the internet using Google’s custom search API. Before using this class you must first create a Google account (link) and generate an API key (link).

Unfortunately, this code cannot be used directly from a Silverlight application because of security limitations (see cross domain policy). If you need this capability in a Silverlight web application I would suggest performing the search via an ASP.NET proxy (license permitting).

In this sample, I called the Google search API from a WPF application, but it would also work in a WinForms or ASP.NET application as suggested above.

Here is the XAML of the Window that displays the results of the Google search.

<Window x:Class="ESRI.PrototypeLab.MapServiceHarvester.MainWindow"
       xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
       xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
       Height="600"
       Width="800"
       >
    <Grid>
<DataGrid x:Name="DataGridResults" AutoGenerateColumns="True" />
    </Grid>
</Window>

Here is the code behind that makes that performs the internet search and displays the results in the datagrid defined above.

public partial class MainWindow : Window {
    public MainWindow() {
        InitializeComponent();

        this.Loaded += (s, e) => {
            GoogleSearch search = new GoogleSearch() {
                Key = "<enter your key here>",
                CX = "013036536707430787589:_pqjad5hr1a"
            };
            search.SearchCompleted += (a, b) => {
                this.DataGridResults.ItemsSource = b.Response.Items;
            };
            search.Search("gis");
        };
    }
}

And here is the class that calls Google’s custom search API, desterilizes the JSON response and returns the result as single .NET objects.

public class GoogleSearch {
    public GoogleSearch() {
        this.Num = 10;
        this.Start = 1;
        this.SafeLevel = SafeLevel.off;
    }
    //
    // PROPERTIES
    //
    public string Key { get; set; }
    public string CX { get; set; }
    public int Num { get; set; }
    public int Start { get; set; }
    public SafeLevel SafeLevel { get; set; }
    //
    // EVENTS
    //
    public event EventHandler<SearchEventArgs> SearchCompleted;
    //
    // METHODs
    //
    protected void OnSearchCompleted(SearchEventArgs e) {
        if (this.SearchCompleted != null) {
            this.SearchCompleted(this, e);
        }
    }
    public void Search(string search) {
        // Check Parameters
        if (string.IsNullOrWhiteSpace(this.Key)) {
            throw new Exception("Google Search 'Key' cannot be null");
        }
        if (string.IsNullOrWhiteSpace(this.CX)) {
            throw new Exception("Google Search 'CX' cannot be null");
        }
        if (string.IsNullOrWhiteSpace(search)) {
            throw new ArgumentNullException("search");
        }
        if (this.Num < 0 || this.Num > 10) {
            throw new ArgumentNullException("Num must be between 1 and 10");
        }
        if (this.Start < 1 || this.Start > 100) {
            throw new ArgumentNullException("Start must be between 1 and 100");
        }
                            
        // Build Query
        string query = string.Empty;
        query += string.Format("q={0}", search);
        query += string.Format("&key={0}", this.Key);
        query += string.Format("&cx={0}", this.CX);
        query += string.Format("&safe={0}", this.SafeLevel.ToString());
        query += string.Format("&alt={0}", "json");
        query += string.Format("&num={0}", this.Num);
        query += string.Format("&start={0}", this.Start);
           
        // Construct URL
        UriBuilder builder = new UriBuilder() {
            Scheme = Uri.UriSchemeHttps,
            Host = "www.googleapis.com",
            Path = "customsearch/v1",
            Query = query
        };

        // Submit Request
        WebClient w = new WebClient();
        w.DownloadStringCompleted += (a, b) => {
            // Check for errors
            if (b == null) { return; }
            if (b.Error != null) { return; }
            if (string.IsNullOrWhiteSpace(b.Result)) { return; }

            // Desearealize from JSON to .NET objects
            Byte[] bytes = Encoding.Unicode.GetBytes(b.Result);
            MemoryStream memoryStream = new MemoryStream(bytes);
            DataContractJsonSerializer dataContractJsonSerializer =
new DataContractJsonSerializer(typeof(GoogleSearchResponse));
            GoogleSearchResponse googleSearchResponse =
dataContractJsonSerializer.ReadObject(memoryStream) as GoogleSearchResponse;
            memoryStream.Close();

            // Raise Event
            this.OnSearchCompleted(
                new SearchEventArgs() {
                    Response = googleSearchResponse
                }
            );
        };
        w.DownloadStringAsync(builder.Uri);

    }
}

public enum SafeLevel { off, medium, high }

public class SearchEventArgs : EventArgs {
    public GoogleSearchResponse Response { get; set; }
}

[DataContract]
public class GoogleSearchResponse {
    [DataMember(Name = "kind")]
    public string Kind { get; set; }
    [DataMember(Name = "url")]
    public Url Url { get; set; }
    [DataMember(Name = "queries")]
    public Queries Queries { get; set; }
    [DataMember(Name = "context")]
    public Context Context { get; set; }
    [DataMember(Name = "items")]
    public List<Item> Items { get; set; }
}

[DataContract]
public class Url {
    [DataMember(Name = "type")]
    public string Type { get; set; }
    [DataMember(Name = "template")]
    public string Template { get; set; }
}

[DataContract]
public class Queries {
    [DataMember(Name = "nextPage")]
    public List<Page> NextPage { get; set; }
    [DataMember(Name = "request")]
    public List<Page> Request { get; set; }
}

[DataContract]
public class Page {
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "totalResults")]
    public int Request { get; set; }
    [DataMember(Name = "searchTerms")]
    public string SearchTerms { get; set; }
    [DataMember(Name = "count")]
    public int Count { get; set; }
    [DataMember(Name = "startIndex")]
    public int StartIndex { get; set; }
    [DataMember(Name = "inputEncoding")]
    public string InputEncoding { get; set; }
    [DataMember(Name = "outputEncoding")]
    public string OutputEncoding { get; set; }
    [DataMember(Name = "safe")]
    public string Safe { get; set; }
    [DataMember(Name = "cx")]
    public string CX { get; set; }
}

[DataContract]
public class Context {
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "facets")]
    public List<List<Facet>> Facets { get; set; }
}

[DataContract]
public class Facet {
    [DataMember(Name = "label")]
    public string Label { get; set; }
    [DataMember(Name = "anchor")]
    public string Anchor { get; set; }
}

[DataContract]
public class Item {
    [DataMember(Name = "kind")]
    public string Kind { get; set; }
    [DataMember(Name = "title")]
    public string Title { get; set; }
    [DataMember(Name = "htmlTitle")]
    public string HtmlTitle { get; set; }
    [DataMember(Name = "link")]
    public string Link { get; set; }
    [DataMember(Name = "displayLink")]
    public string DisplayLink { get; set; }
    [DataMember(Name = "snippet")]
    public string Snippet { get; set; }
    [DataMember(Name = "htmlSnippet")]
    public string HtmlSnippet { get; set; }
    [DataMember(Name = "cacheId")]
    public string CacheId { get; set; }
    //[DataMember(Name = "pagemap")] *** Cannot deserialize JSON to .NET! ***
    //public Pagemap Pagemap { get; set; }
}

[DataContract]
public class Pagemap {
    [DataMember(Name = "metatags")]
    public List<Dictionary<string, string>> Metatags { get; set; }
}

[DataContract]
public class Metatag {
    [DataMember(Name = "creationdate")]
    public string Creationdate { get; set; }
    [DataMember(Name = "moddate")]
    public string Moddate { get; set; }
}

And lastly, here is the result.

image

The only issue I had with this sample was trying to deserialize the JSON associated with the “pagemap” property (see commented out code above). Any tips would be appreciated.

18 comments:

  1. Hi, Thanks For code,

    But its give error 400 bad request.

    ReplyDelete
  2. Did you generate your own key?

    ReplyDelete
  3. Is it possible to use this code from a console application ? I am trying to make a google search and just take the first URL returned by google. Any suggestions would be appreciated.

    ReplyDelete
  4. Sure!
    Use this code:

    GoogleSearch search = new GoogleSearch() {
    Key = "",
    CX = "013036536707430787589:_pqjad5hr1a"
    };
    search.SearchCompleted += (a, b) => // here goes your your method to write the results to the console;
    search.Search("gis");

    ReplyDelete
  5. Why you don't attach a complete solution ?

    ReplyDelete
  6. Hello, your post help me a lot.
    But, i need to make my searh in google.es domain, where i can define the search domain to use in my queryes?-
    Best regards

    ReplyDelete
  7. Other question, how can i obtain the searh result, in the same order thats browser show?.
    Example:
    Make a query in www.google.es browser, the result is:
    1.- Site 1
    2.- Site 2
    3.- Site 3

    Using this code, the resilt is
    1.- Site 3
    2.- Site 1
    3.- Site 2

    How can i obtain the same result that query browser?

    ReplyDelete
  8. This was a great help! One thing I ran into was that certain characters, such as #, and the query wouldn't go through at all. Using System.Web.HttpUtility.UrlEncode fixed that for me.

    ReplyDelete
  9. how to get result more then 10 ?

    ReplyDelete
  10. I generated my own key but i'm still having the error "The remote server returned an error. (400)Bad request.", help please!

    ReplyDelete
    Replies
    1. Did you ever figure this out? I am having the same issue

      Delete
  11. It displays only First 10 results, how to implement navigation to get next 10 results and so on

    ReplyDelete
  12. How to get more than 10 results? as per google api agreement, we can get 1000 results per query. How to get that?

    ReplyDelete
  13. How do i hyperlink the "link" results

    ReplyDelete
  14. This is awesome code and well written, worked first time.

    Did you ever get the pagemap property to work?

    Thanks

    ReplyDelete
  15. How to save this? I can see there are 3 code here and I dont know how to save into what files format to make it works.

    ReplyDelete
  16. Additional information: There was an error deserializing the object of type VodafoneSearchDemo.GoogleSearchResponse. Encountered unexpected character '<'.

    ReplyDelete
  17. Hello! I can see only the first 10 records.all results in the DataGrid without paging Is it possible to show ? tnx

    ReplyDelete