Friday, May 8, 2015

Monthly CO2 Levels Above 400ppm



For the first time since modern records began, monthly mean carbon dioxide levels were above 400 parts per million (ppm), as illustrated by the NOAA image below. NOAA just released the mean global carbon dioxide level for March 2015, which was 400.83 ppm.
Arctic Ocean hit hard

Carbon dioxide concentrations can be especially high, i.e. well over 410 ppm, at higher latitudes of the Northern Hemisphere, as illustrated by the NOAA image below. This can contribute to very high temperature anomalies over the Arctic Ocean and thus increase the risk of huge amounts of methane erupting from the Arctic Ocean seafloor. 
Image contributed by Harold Hensel
Since the start of the Industrial Revolution, carbon dioxide levels have risen non-linearly, as illustrated by the image below.

Need for Comprehensive and Effective Action

As many posts at this blog have warned, emissions by people and the numerous feedbacks are threatening to push Earth into runaway global warming.

This calls for comprehensive and effective action to - among other things - reduce atmospheric carbon dioxide levels back to 280 ppm, as illustrated by the image below and as further discussed at the policies proposed as part of the Climate Plan.


How best to get back to 280 ppm? 

The Climate Plan calls for restoration of greenhouse gas levels in oceans and atmosphere to their long term average by 2100. In the Climate Plan, multiple lines of action are proposed to work simultaneously, in parallel and separately in their implementation, yet complementary in their impact.


One line of action is to cut emissions by 80% by the year 2020. To achieve this, the Climate Plan advocates implementation of local feebates. Especially important are fees on sales of fuel, while the resulting revenues are best used to fund rebates on products sold locally that further help speed up the shift to clean energy.

Without further action, much of the carbon dioxide that has been emitted will stay in the atmosphere for hundreds, if not thousands of years. Therefore, further lines of action are needed, including removal of carbon dioxide from the atmosphere and oceans, with the carbon being safely stored.

For the long term average of 280 ppm to be achieved in 2100, large amounts of atmospheric carbon dioxide must also be removed and safely stored annually. How much must be removed? The period from 2015 to 2100 has 85 years, so bringing down carbon dioxide from 400 ppm to 280 ppm over that period works out to an annual removal of 1.41 ppm. By comparison, this is slightly less than the annual growth in carbon dioxide levels as caused by people since 1959, which is on average 1.47 ppm. Assuming that emissions will not be cut quickly enough to avoid further build up of carbon dioxide in the atmosphere in the near future, annual removal will need to be somewhat more, so 1.47 ppm looks like a good target for now, precisely because it equals past emissions.


The Climate Plan thus proposes that each nation will contribute to the necessary annual 1.47 ppm removal with a share that reflects its past emissions. The image below gives an idea of past emissions. Note that the image only shows emissions up to the year 2011 and that they exclude land use change and forestry emissions. Furthermore, the image shows emissions based on where products were produced. Much of the rise in emissions is the result of products that were produced in Asia, yet many of these products were consumed in Europe and North America. Therefore, graphs based on emissions where products were consumed would paint a somewhat different picture. The Climate Plan proposes that a nation's contributions to carbon dioxide removal (from oceans and atmosphere) will reflect its past emissions based on where products were consumed.


The Climate Plan advocates separate lines of action, i.e. greenhouse gas removal next to emission cuts and further action. Keeping action on different types of pollution separate and calling for local action in each nation further helps avoid that progress elsewhere is pointed at by a nation as an excuse to delay the necessary action on a specific type of pollution in that nation.

As said, the Climate Plan therefore calls for fees to be added on sales of polluting products where they are consumed (as opposed to where they are produced). Each nation is further expected to take steps to contribute its share to the 1.47 ppm carbon dioxide that needs to be removed from the atmosphere annually. Additionally, carbon dioxide needs to be removed from the oceans.

The most important thing each nation can do in the lead-up to the upcoming U.N. climate conference in Paris is to accept these commitments. How each nation and local community does achieve targets is best decided locally, provided that each nation and each local community does indeed reach its targets, and this follows from this commitment.

One reason why local feebates are recommended is that they can focus on achieving local targets for a specific pollutant. Local feebates allow communities to quickly adjust the height of the fees, where a local community threatens to fail reaching a target. Such a local focus does not preclude action being beneficial elsewhere as well. Indeed, the same feebate can work for multiple pollutants and on multiple lines of action. In this sense, locally implemented feebates often work complementary. As an example, the feebates pictured below will help remove carbon dioxide from the atmosphere and oceans, while they will also help cut emissions of carbon dioxide, methane, soot and nitrous oxide.




Further background

- Climate Plan
http://arctic-news.blogspot.com/p/plan.html

- Feebates
http://feebates.blogspot.com/p/feebates.html

- Policies
http://arctic-news.blogspot.com/p/policies.html

- Action
http://arctic-news.blogspot.com/p/action.html



The Climate Plan calls for: - 80% emission cuts by 2020, for each type of pollutant, in each location and best managed...
Posted by Sam Carana on Saturday, May 9, 2015

Wednesday, May 6, 2015

4 Web Scraping Tools To Save You Time On Data Extraction

Either you are working on a product website, struggling to add live data feed to your app or merely need to pull out a huge amount of online data for analysis, an accurate web scraping tool can save you loads of time and keep you sane. Here are four powerful web scraping tools to save you from copy-pasting or spending time on writing your own scripts.

1. Uipath

Either you are working on a product website, struggling to add live data feed to your app or merely need to pull out a huge amount of online data for analysis, an accurate web scraping tool can save you loads of time and keep you sane. Here are four powerful web scraping tools to save you from copy-pasting or spending time on writing your own scripts.

1. Uipath

Uipath specializes in developing various process automation software including web scraping and screen scraping software for desktop and web. Uipath web scraper is perfect for non-coders and easily surpasses most common data extraction challenges including page navigation, digging through flash and even scraping PDF files. All you need to do is open the web scraping wizard and simply highlight the data you need to extract. The tool will scrape all the data following this pattern at all pages you’ve chosen and sort it accordingly. You can add as many items for scraping as you like and have them sorted in respective columns. As a result, you receive a neat Excel or CSV document with all the data eliminated from duplicates.

Moreover, Uipath isn’t just about scraping. This software can be used not only for extracting data, but to manipulate the interface of another app, thus establishing data transfers among the two of them. Basically, this tool could be used to conduct any repetitive task a human could do, yet much faster and with higher accuracy.

Pros: You can automate form filling, clicking buttons, navigation etc. Uipath scraper is impressively accurate, fast and simple to use. It “reads” all types of data on screen (JS, HTML, Silverlight and more), plus you can train the software to emulate human actions of various complexity.

Cons: Premium software runs at a premium price. Uipath is an affordable professional solution, but may be a bit too pricey for personal use.

2. Import.io

Data Extraction

Either you are working on a product website, struggling to add live data feed to your app or merely need to pull out a huge amount of online data for analysis, an accurate web scraping tool can save you loads of time and keep you sane. Here are four powerful web scraping tools to save you from copy-pasting or spending time on writing your own scripts.

1. Uipath

Uipath specializes in developing various process automation software including web scraping and screen scraping software for desktop and web. Uipath web scraper is perfect for non-coders and easily surpasses most common data extraction challenges including page navigation, digging through flash and even scraping PDF files. All you need to do is open the web scraping wizard and simply highlight the data you need to extract. The tool will scrape all the data following this pattern at all pages you’ve chosen and sort it accordingly. You can add as many items for scraping as you like and have them sorted in respective columns. As a result, you receive a neat Excel or CSV document with all the data eliminated from duplicates.

Moreover, Uipath isn’t just about scraping. This software can be used not only for extracting data, but to manipulate the interface of another app, thus establishing data transfers among the two of them. Basically, this tool could be used to conduct any repetitive task a human could do, yet much faster and with higher accuracy.

Pros: You can automate form filling, clicking buttons, navigation etc. Uipath scraper is impressively accurate, fast and simple to use. It “reads” all types of data on screen (JS, HTML, Silverlight and more), plus you can train the software to emulate human actions of various complexity.

Cons: Premium software runs at a premium price. Uipath is an affordable professional solution, but may be a bit too pricey for personal use.

2. Import.io

Import.io offers you a free desktop app to help you scrap all the data you need from an unlimited amount of web pages. The service treats each page as a potential data source to generate API from. If the page you’ve submitted has been previously processed, you can access its API and get some of the data. In other case, Import.io will guide you through the process of creating the scraping matrix by building connectors (for navigation) or extractors (to pull out the needed data). Afterwards, you submit a request for extraction and it’s typically processed within 24 hours. All the data is private and you can schedule auto refreshments at any chosen period of time.

Pros: The service is easy-to-use with no tech skills needed. It can  pages with data (those that needed login/pass), plus it’s free. Minimalistic effective design and simple navigation comes along.

Cons: Improt.io has hard times navigating through combinations of javascript/POST and cannot navigate from one page to another (e.g. click next, second page etc).  Sometimes, it takes over 24 hours to receive the report.  Besides, it’s a browser-only app, non-compatible with other applications.

3. Kimono

Either you are working on a product website, struggling to add live data feed to your app or merely need to pull out a huge amount of online data for analysis, an accurate web scraping tool can save you loads of time and keep you sane. Here are four powerful web scraping tools to save you from copy-pasting or spending time on writing your own scripts.

1. Uipath

Uipath specializes in developing various process automation software including web scraping and screen scraping software for desktop and web. Uipath web scraper is perfect for non-coders and easily surpasses most common data extraction challenges including page navigation, digging through flash and even scraping PDF files. All you need to do is open the web scraping wizard and simply highlight the data you need to extract. The tool will scrape all the data following this pattern at all pages you’ve chosen and sort it accordingly. You can add as many items for scraping as you like and have them sorted in respective columns. As a result, you receive a neat Excel or CSV document with all the data eliminated from duplicates.

Moreover, Uipath isn’t just about scraping. This software can be used not only for extracting data, but to manipulate the interface of another app, thus establishing data transfers among the two of them. Basically, this tool could be used to conduct any repetitive task a human could do, yet much faster and with higher accuracy.

Pros: You can automate form filling, clicking buttons, navigation etc. Uipath scraper is impressively accurate, fast and simple to use. It “reads” all types of data on screen (JS, HTML, Silverlight and more), plus you can train the software to emulate human actions of various complexity.

Cons: Premium software runs at a premium price. Uipath is an affordable professional solution, but may be a bit too pricey for personal use.

2. Import.io

Import.io offers you a free desktop app to help you scrap all the data you need from an unlimited amount of web pages. The service treats each page as a potential data source to generate API from. If the page you’ve submitted has been previously processed, you can access its API and get some of the data. In other case, Import.io will guide you through the process of creating the scraping matrix by building connectors (for navigation) or extractors (to pull out the needed data). Afterwards, you submit a request for extraction and it’s typically processed within 24 hours. All the data is private and you can schedule auto refreshments at any chosen period of time.

Pros: The service is easy-to-use with no tech skills needed. It can  pages with data (those that needed login/pass), plus it’s free. Minimalistic effective design and simple navigation comes along.

Cons: Improt.io has hard times navigating through combinations of javascript/POST and cannot navigate from one page to another (e.g. click next, second page etc).  Sometimes, it takes over 24 hours to receive the report.  Besides, it’s a browser-only app, non-compatible with other applications.

3. Kimono

Kimono is a popular web scraper among app developers who prefer to power up their products with live data and no additional code. It saves you tons of time when you need to fill up your app with mashing data. Install Kimono Browser bookmarklet; highlight page elements you need to and provide some positive/negative examples to train the tool. After labeling all the data you can download it in CSV/JSON/a web endpoint format. The APIs created for your pages are stored in the cloud and you can run them on schedule. So far, Kimono is free to use with pro and enterprise solutions to be launched soon.

Pros: The tool works pretty fast and works great with scraping newsfeeds and prices. The data is rather accurate.

Cons: No page navigation available and you need to spend quite a lot of time to train Kimono before it starts to pull out the multi items data accurate enough. In general, I’d say Kimono is more of an app mash-ups creator than a full-scale web scraper.

4. Screen Scraper

Either you are working on a product website, struggling to add live data feed to your app or merely need to pull out a huge amount of online data for analysis, an accurate web scraping tool can save you loads of time and keep you sane. Here are four powerful web scraping tools to save you from copy-pasting or spending time on writing your own scripts.

1. Uipath

Uipath specializes in developing various process automation software including web scraping and screen scraping software for desktop and web. Uipath web scraper is perfect for non-coders and easily surpasses most common data extraction challenges including page navigation, digging through flash and even scraping PDF files. All you need to do is open the web scraping wizard and simply highlight the data you need to extract. The tool will scrape all the data following this pattern at all pages you’ve chosen and sort it accordingly. You can add as many items for scraping as you like and have them sorted in respective columns. As a result, you receive a neat Excel or CSV document with all the data eliminated from duplicates.

Moreover, Uipath isn’t just about scraping. This software can be used not only for extracting data, but to manipulate the interface of another app, thus establishing data transfers among the two of them. Basically, this tool could be used to conduct any repetitive task a human could do, yet much faster and with higher accuracy.

Pros: You can automate form filling, clicking buttons, navigation etc. Uipath scraper is impressively accurate, fast and simple to use. It “reads” all types of data on screen (JS, HTML, Silverlight and more), plus you can train the software to emulate human actions of various complexity.

Cons: Premium software runs at a premium price. Uipath is an affordable professional solution, but may be a bit too pricey for personal use.

2. Import.io

Import.io offers you a free desktop app to help you scrap all the data you need from an unlimited amount of web pages. The service treats each page as a potential data source to generate API from. If the page you’ve submitted has been previously processed, you can access its API and get some of the data. In other case, Import.io will guide you through the process of creating the scraping matrix by building connectors (for navigation) or extractors (to pull out the needed data). Afterwards, you submit a request for extraction and it’s typically processed within 24 hours. All the data is private and you can schedule auto refreshments at any chosen period of time.

Pros: The service is easy-to-use with no tech skills needed. It can  pages with data (those that needed login/pass), plus it’s free. Minimalistic effective design and simple navigation comes along.

Cons: Improt.io has hard times navigating through combinations of javascript/POST and cannot navigate from one page to another (e.g. click next, second page etc).  Sometimes, it takes over 24 hours to receive the report.  Besides, it’s a browser-only app, non-compatible with other applications.

3. Kimono

Kimono is a popular web scraper among app developers who prefer to power up their products with live data and no additional code. It saves you tons of time when you need to fill up your app with mashing data. Install Kimono Browser bookmarklet; highlight page elements you need to and provide some positive/negative examples to train the tool. After labeling all the data you can download it in CSV/JSON/a web endpoint format. The APIs created for your pages are stored in the cloud and you can run them on schedule. So far, Kimono is free to use with pro and enterprise solutions to be launched soon.

Pros: The tool works pretty fast and works great with scraping newsfeeds and prices. The data is rather accurate.

Cons: No page navigation available and you need to spend quite a lot of time to train Kimono before it starts to pull out the multi items data accurate enough. In general, I’d say Kimono is more of an app mash-ups creator than a full-scale web scraper.

4. Screen Scraper

Screen scraper is pretty neat and tackles a lot of difficult tasks including navigation and precise data extractions, however it requires a bit of programming/tokenization skills if you’d like to run it super smooth. Launch the software, add a proxy, start recording the list of your actions and creating extracting patterns (some coding required). Works great with HTML and Javascript, however you should test it with Citrix and other platforms. Basically, screen scraper helps you writing simple web scraping scripts and lets you download the extracted data in txt/csv/excel format.

Pros: When set correctly, there’s no data extraction tasks Screen scraper fails to handle.

Cons: The tool is pricey and you’ll have to go through documentation and have basic coding skills to use it.

Source: http://tech.co/4-web-scraping-tools-save-time-data-extraction-2015-03

Thursday, April 30, 2015

Web Data Extraction Solutions for business Automation

Your business today is driven purely and solely by information. This vital component is carved out from data scraped from relevant sources, cleansed and compiled to form the crux of your enterprise analytics plan. Therefore, following a manual practice for Data Extraction often makes it prone to errors which may be detrimental to the health of your business. Various Web Data Extraction Solutions therefore are made available which help to not only automate Website Data Scraping, but in the process also help in the automation of several business processes.  Let us take a look at some of these business processes:

Execute Data Mining

One of the most obvious usages of the data extractor tools is the function of Web Data Mining, which, if done through manual processes is neither cost-effective nor accurate. Therefore, data extraction solutions provide simple and convenient point and click data extraction interfaces. Moreover, these do not require any additional programming knowledge.

Validate Data accuracy

Web Site Data scraping tools use advanced technology to not only extract data, but also validate their accuracy. This is particularly beneficial for businesses which are involved in background screening and credit reporting activities. Automating the process of validating records helps in increasing turnaround time of the business and ensures accuracy, two of the most crucial criteria for achieving success in their line of business.

Be Price Wise

Organizations are always involved in studying the price dynamics in order to better understand the challenges and areas of opportunities. Your awareness on these helps you to provide more competitive pricing for your products and services. Data extraction solutions are equipped with pricing intelligence technology that helps to collect data on the price that your customer’s expect, their feedback on your products and services and helps you develop insights on products being launched by competitors, their price and availability. This automated process therefore ensures the following things for your business:

•    Increased market – share

•    Enhanced product strategy

•    Informed decisions

Be Compliance Ready

If yours is a financial services firm, you must have often found it to be a major challenge to keep yourself abreast of the compliance and risk factors. Tracking the myriad watch lists, sanction lists and federal and state regulations, available on the web is not only time taking but also an expensive endeavor. The automated Web Data Mining tools ensure that you are now able to do this without impacting the time and money aspect of your business. You are now ensured of compliance with all regulations and sanction lists that are updated on a regular basis. Consequently, your business can also breathe easy with a reduced exposure to financial fraud and identity theft.

Easy Access to Customer Feedback

Your customers are in a way the real owners of your business. They define the way you need to design your product strategy. It is therefore crucial that you need to listen to what they have to say.  The automated Web extraction solutions with their ability to tap into several relevant sources help you to get access to customer sentiments and feedback on your products and services. This is a vital aspect in your organization's growth as it helps you to tackle three important factors:

•    Assimilate positive or negative vibes on your newly launched product or service which might require you to revisit your product strategy

•    Provide immediate attention to issues, if any, with any particular product

Create a trend of aggregated customer sentiments, for analysis

Source: http://scraping-solutions.blogspot.in/2014_07_01_archive.html

Tuesday, April 28, 2015

Scraping a website from a windows service

Question

Hi there.  I have a windows forms application that scrapes a website to retrieve some data.  I would like to implement the same functionality as a windows service.  The reason for this is to allow the program to run 24/7 without  having a user signed in. 

To that end, my current version of the program uses a web browser control (system.windows.forms.webbrowser) to navigate the pages, click the buttons, allow scripts to do their thing, etc.  I cannot figure out a way to do the same without the web browser control, but the web browser control cannot be instantiated in a windows service (because there is no user interface in a web service).

Does anyone have any brilliant ideas on how to get around this?

Thank you very much!

Answers

Hi Andy,

There is a tool which could let you manipulate anything you want on the website. This agile HTML parser builds a read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to what proposes System.Xml, but for HTML documents (or streams). More information, please check:

http://htmlagilitypack.codeplex.com/

Have a nice day.

Best regards

All replies

You are not telling if you are using a .NET Express edition or not

You are not telling which Framework

You are not realy saying what data you are getting from the web site.

So

I made an example of service that work on any Studio edition (including the Express)

to install it, I supposed that you have at least the Framework2, so you will use something similar to:

    %SystemRoot%\Microsoft.NET\Framework\v2.0.50727\installutil /i C:\Test\MyWindowService\MyWindowService\bin\Release\MyWindowService.exe

In the example, I supposed that you are downloading some file from the site

You will need a reference to Windows.Form for the timer


Imports System.ServiceProcess

Imports System.Configuration.Install

Public Class WindowsService : Inherits ServiceBase

  Private Minute As Integer = 60000

  Private WithEvents Timer As New Timer With {.Interval = 30 * Minute, .Enabled = True}

  Public Sub New()

    Me.ServiceName = "MyService"

    Me.EventLog.Log = "Application"

    Me.CanHandlePowerEvent = True

    Me.CanHandleSessionChangeEvent = True

    Me.CanPauseAndContinue = True

    Me.CanShutdown = True

    Me.CanStop = True

  End Sub


  Private Sub Timer_Tick(ByVal sender As Object, ByVal e As System.EventArgs) Handles Timer.Tick

    If IO.File.Exists("C:\MyPath.Data") Then IO.File.Delete("C:\MyPath.Data")

    My.Computer.Network.DownloadFile("http://MyURL.com", "C:\MyPath.Data", "MyUserName", "MyPassword")

    'Do Something with the data downloaded

  End Sub

End Class

<Microsoft.VisualBasic.HideModuleName()> _

Module MainModule

  Public TheServiceName As String

  Public Sub main()

    Dim TheServiceApplication As New WindowsService

    TheServiceName = TheServiceApplication.ServiceName

    ServiceBase.Run(TheServiceApplication)

  End Sub

End Module

<System.ComponentModel.RunInstaller(True)> _

Public Class WindowsServiceInstaller : Inherits Installer

  Public Sub New()

    Dim serviceProcessInstaller As ServiceProcessInstaller = New ServiceProcessInstaller()

    Dim serviceInstaller As ServiceInstaller = New ServiceInstaller()

    serviceProcessInstaller.Account = ServiceAccount.LocalSystem

    serviceProcessInstaller.Username = Nothing

    serviceProcessInstaller.Password = Nothing

    serviceInstaller.DisplayName = "My Windows Service"

    serviceInstaller.StartType = ServiceStartMode.Automatic

    serviceInstaller.ServiceName = TheServiceName

    Me.Installers.Add(serviceProcessInstaller)

    Me.Installers.Add(serviceInstaller)

  End Sub

End Class



 Hello Andy,

Thanks for your post.

What do you want to scrape from the page? HttpWebRequest class ans WebClient class may be what you need. More information, please check:

The HttpWebRequest class provides support for the properties and methods defined in WebRequest and for additional properties and methods that enable the user to interact directly with servers using HTTP.

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx

The WebClient class provides common methods for sending data to or receiving data from any local, intranet, or Internet resource identified by a URI

http://msdn.microsoft.com/en-us/library/system.net.webclient.aspx

If you have any concenrs, please feel free to follow up.

Best regards



Hi Andy,

What about this problem on your side now? If you have any concerns, please feel free to follow up.

Have a nice day.

Best regards



Hi Andy,

When you come back, if you need further assistance about this issue, please feel free to let us know. We will continue to work with this issue.

Have a nice day.

Best regards



Thank you for the reply. Sorry it has taken me so long to respond.  I did not receive any notification that someone had replied!

I am using Visual Studio 2010 Ultimate Edition and the .NET framework 4.0.  Actually, I am upgrading some old code written in VB 6.0, but I can use the latest and greatest thats available.

The application uses a browser control to go to the page, fill in values, click on UI elements, read the HTML that returns, etc.  The purpose of the application is to collection useful information regularily/automatically.

I know how to create a web service, but using the web control in such a service is problematic because the web browser control was meant to be placed on a windows form.  I am not able to create a new instance of it in a project designated as a windows service.



Andy

Thank you for the reply. Sorry it has taken me so long to respond.  I did not receive any notification that someone had replied!

I thought a web request was for web services (retrieving information from them).  I am trying to retreive useful information from a website designed for interaction by a human, such as selecting items from lists and clicking buttons.   I currently use a web browser control to programmatically do what a person would do and get the pages back which in turn get parsed.

Andy



Hi Andy,

There is a tool which could let you manipulate anything you want on the website. This agile HTML parser builds a read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to what proposes System.Xml, but for HTML documents (or streams). More information, please check:

http://htmlagilitypack.codeplex.com/

Have a nice day.

Best regards



Thanks for the suggestion.  I will go to that link and see if it will work.  I will update this post with what I find.

I am writing to check the status of the issue on your side. Would you mind letting us know the result of the suggestions? If you have any concerns, please feel free to follow up.

Have a nice day.



Best regards

Hi Liliane

Thank for the follow up reply.  I don't have an answer as of yet.  Implementing this is going to take time and I haven't been given the go-ahead by my boss to spend the time to pursue it.



Hi Andy,

Never minde. You could have a try when you feel free. If you have any further questions about this issue, please feel free to let us know. We will continue to work with you on this issue.

Have a nice day.

Best regards

Source: https://social.msdn.microsoft.com/Forums/vstudio/en-US/f5d565b1-236b-43c2-90c7-f5cc3b2c341b/scraping-a-website-from-a-windows-service

Sunday, April 26, 2015

Methane levels as high as 2845ppb

Methane levels as high as 2845 parts per billion (ppb) were recorded on April 25, 2015, as illustrated by the image below.


This is an extremely high peak. The average daily peak in 2015 until May 1 was 2371 ppb, while the highest daily mean ranged from 1807 ppb (January 10) to 1829 ppb (April 22). Daily peaks and daily highest mean levels in 2015 are shown on the image below.



These peaks are much higher than they were in previous years, as illustrated by the image below, from an earlier post and showing the average highest peak readings in 2013 and 2014 at selected altitudes..


Peak readings in above image are averages over April 2013 and April 2014. On specific days, peak readings could be much higher, e.g. on April 28, 2014, methane levels were recorded as high as 2551 ppb at 469 mb. As said, methane levels as high as 2845 ppb were recorded on April 25, 2015, while the average peak for the first four months of 2015 was 2371 ppb, and this average was calculated from peaks across altitudes.

The table below shows the altitude equivalents in mb (millibar) and feet.
56,925 ft44,689 ft36,850 ft30,569 ft25,543 ft19,819 ft14,383 ft8,367 ft1,916 ft
74 mb147 mb218 mb293 mb367 mb469 mb586 mb742 mb945 mb

Peak levels in April appear to be rising strongly each year, following higher peak readings during previous months, especially at higher altitudes, i.e. especially the Arctic Ocean. It appears that much of the additional methane originating from the higher latitudes of the Northern Hemisphere has moved closer to the equator over the past few months, and is now accumulating at higher altitudes over the continents on the Northern Hemisphere, i.e. Asia, Europe, North America and north Africa.

Further analysis of the rise in global mean methane levels appears to confirm the above. The image below shows methane levels on April 22, over three years. While there appears to be little or no rise in mean methane levels at low altitudes, the rise is quite profound at higher altitudes.  

[ click on image to enlarge ]
Things look set to get worse. As shown by the image below, from an earlier post, global methane levels have risen sharply from a low of 723 ppb in 1755. Mean methane levels were as high as 1839 ppb in 2014. That's a rise of more than 254%.
As that post concluded a year ago, it appears that the rise of methane in the atmosphere is accelerating. What can we expect? As temperatures can be expected to continue to rise and as feedbacks start to kick in, this may well constitute a non-linear trend. The image below shows a polynomial trend that is contained in IPCC AR5 data from 1955 to 2011, pointing at methane reaching mean global levels higher than 3000 ppb by the year 2030. If methane starts to erupt in large quantities from clathrates underneath the seafloor of the Arctic Ocean, this may well be where we are heading. 




The situation is dire and calls for comprehensive and effective action, as discussed at the Climate Plan.



The 2845 ppb recorded on April 25, 2015, is an extremely high peak. The average daily peak in 2015 until now was 2372...
Posted by Sam Carana on Monday, April 27, 2015

Saturday, April 25, 2015

Data Mining and Market Research

Online market research attributes to success and growth of many businesses. Online market research in simple terms we can say it is the learning of current and the latest market situations which involve surveys, web and data mining modules. To date research by use of the internet it is very important since it depends on data gathered from internet services and then one can recognize that market research keeps the business successful.

A number of managers in small businesses have a mental deem in that online market research is obligatory to big or larger companies. For true you will understand that whether the businesses is medium, large or small actually need online marketing research and this is a reason why the significance of the process and allegation will approve the targeted and potential clients. In this case Data mining progression is employed to streamline on what targeted and potential clientele needs. Areas where data mining is used:

Preferences. In any given service or product, you will learn what a customer looking for and how your product or service is different from other competitors. By use of Data mining you will be able to determine the customer preferences and you will be able to modify your services and products meet the customer choice.

Buying patterns. What is known and created for purchasing patterns from different customers. A situation can be that customers try to spend a lot on certain products and little on others. But through Data mining it is easy to understand such purchasing patterns and finally plan the appropriate techniques to be used in the marketing.

Prices. To find out whether the company is selling its products to the clients or not prices are the key factors to take into account. One should understand on the right selling price of the products. Web scrapping is easier to find the suitable pricing.

Source: http://www.loginworks.com/blogs/web-scraping-blogs/data-mining-market-research/

Thursday, April 23, 2015

It's time to 'Do the math' again

By David Spratt

Have we gone mad? A new report released today explains why contemporary climate change policy-making should be characterised as increasingly delusional.

As the deadline approaches for submissions to the Australian government's climate targets process, there is a flurry of submissions and reports from advocacy groups and the Climate Change Authority.

Most of these reports are based on the twin propositions that two degrees Celsius (2°C) of global warming is an appropriate policy target, and that there is a significant carbon budget and an amount of "burnable carbon" for this target, and hence a scientifically-based escalating ladder of emission-reduction targets stretching to mid-century and beyond.

A survey of the relevant scientific literature by David Spratt, "Recount: It's time to 'Do the math' again", published today by Breakthrough concludes that the evidence does not support either of these propositions.

The catastrophic and irreversible consequences of 2°C of warming demand a strong risk-management approach, with a low rate of failure. We should not take risks with the climate that we would not take with civil infrastructure.

There is no carbon budget available if 2°C is considered a cap or upper boundary as per the Copenhagen Accord, rather than a hit-or-miss target which can be significantly exceeded; or if a low risk of exceeding 2°C is required; or if positive feedbacks such as permafrost and other carbon store losses are taken into account.

Effective policy making can only be based on recognising that climate change is already dangerous, and we have no carbon budget left to divide up. Big tipping-point events irreversible on human time scales such as in West Antarctica and large-scale positive feedbacks are already occurring at less than 1°C of warming. It is clear that 2°C of climate warming is not a safe cap.

In reality, 2°C is the boundary between dangerous and very dangerous climate change and 1°C warmer than human civilisation has ever experienced.

In the lead up to the forthcoming Paris talks, policy makers through their willful neglect of the evidence are in effect normalising a 2.5–3°C global warming target.

This evidence in "Recount: It's time to 'Do the math' again" demonstrates that action is necessary at a faster pace than most policy makers conceive is possible.



Related

- It's time to 'Do the math' again
http://www.climatecodered.org/2015/04/its-time-to-do-math-again.html

- RECOUNT - It's time to 'Do the math' again
http://media.wix.com/ugd/148cb0_938b5512abfa4d4e965ec8cc292893f7.pdf

- Two degrees of warming closer than you may think
http://arctic-news.blogspot.com/2015/02/two-degrees-of-warming-closer-than-you-may-think.html

- The real budgetary emergency and the myth of "burnable carbon"
http://arctic-news.blogspot.com/2014/05/the-real-budgetary-emergency-and-the-myth-of-burnable-carbon.html

It's time to 'Do the math' again | by David Spratt http://arctic-news.blogspot.com/2015/04/its-time-to-do-math-again.html

Posted by Sam Carana on Thursday, April 23, 2015