Tag: Software Dev

Zmanim Bug Report from the Land of the Midnight Sun

I was recently contacted by Jan Terje Johansen, a developer at Datek Wireless AS in Norway, with an interesting bug report. Datek uses the Zmanim API (the AstronomicalCalendar base class) to allow their clients (the power company and stadiums) to remotely (via a web interface) control streetlights and stadium lights throughout Norway using a wireless lighting control system that they developed. The zmanim code is used to allow setting the lighting times based on an offset of sunrise/sunset. For the technically curious, they are controlled primarily through GPRS, with SMS as fallback. A version under development uses ZigBee. The bug encountered was that for Tromsoe (Tromsø), Norway, as well as other areas within the Arctic Circle that experience the midnight sun, from May 13th to May 17th (the date of last rise of the season in Tromsoe), the zmanim API produced correct sunrise/set times, but the date component (The API returns all times as Java Dates, something that might change with v2.0 of the API that will target JDK 7 to take advantage of JSR-310 Date and Time API) was a day off. For non-automated systems, the date component is not important, but in their case it would cause the lights to go on/off on the wrong day. Jan provided a suggested patch that worked well. The actual fix I used was slightly different because I took advantage of the time spent on fixing the bug to refactor and simplify the code. This change as well as a few other changes are part of the Zmanim 1.1 beta release that will likely be released as a final release in a few days. Jan mentioned that:

“IMHO your API is easily the best and most accurate Open Source sunset/sunrise API out there”. He continued: “Officially (according to the Norwegian Meteorology Institute), the midnight sunset/sunrise is from May 20 to July 22 in Tromsoe, ie. the complete sun is above the horizon 24 hours. Parts of sun is visible 24 hours a day in Tromsoe from May 18th to July 25th. This is the same as in your calculations.” … “I have tested your calculations against other official midnight sunrise/sunset (part of sun) dates in Northern-Norway (North Cape, Hammerfest, Longyearbyen (78,049762N -15,458252E)) and they are spot on.”

In response to my question regarding his testing of the NOAACalculator versus the USNOCalculator he had an interesting and very practical answer

“We prefer to use USNO calculator as it is more in tune with the sunrise/sunset times printed in most newspapers. You see, our experience is that most users don’t look at the sun to determine sunrise/sunset but read the times in the newspaper. If our times don’t correspond to the printed ones, something is wrong with our system in their mind.”

Zmanim API 1.1 Release Candidate Available

A release candidate (RC1) of the Zmanim API 1.1 is now available on the download page. Changes in this release include a slight clean up of the recently released NOAACalculator code (does not change calculated times), as well as fixes to the date (but not time) of calculations for locations near the arctic circle. This date fix builds on the February release of the API to fix an arctic circle issue and a similar issue encountered when trying to generate zmanim for locations other than the local timezone. Also included in this release is the zmanimAstronomical-1.1.jar, a release that only includes the AstronomicalCalendar class and supporting classes. There was also some code refactoring to make the code easier to maintain. A detailed post will follow (hopefully within the next week or so).

Fix to NOAA Sunrise/Sunset Algorithm

The Zmanim API was developed from the ground up as an API which allows for easy plugging in of different algorithms. The Zmanim API ships with 3 “Calculator” implementations. Two calculators implement the US Naval Observatory’s algorithm, the SunTimesCalculator and the ZmanimCalculator (no longer available). Both produce identical zmanim using slightly different code and are included for comparison. There is also the JSuntimeCalculator (no longer available – see below information on the NOAACalculator), an implementation of the NOAA algorithm by Jonathan Stott. I was recently contacted by Eliezer Bulka who wanted to know why sunrise/sunset times generated by the NOAA algorithm were about 2 minutes off of the sunrise/sunset times generated by the NOAA JavaScript implementation that is the source of the JSuntimeCalculator. To compare apples and apples required modification of the NOAA JavaScript to allow entry of decimal latitude/longitude and changing the output to display seconds. No change was made to the algorithm itself. I then ported the JavaScript directly to Java. This involved nothing more than slight syntax changes between the languages. Once this was done, I noticed that the sun rise/set output from the Java port exactly matched the output of the NOAA JavaScript. Analysis of Jonathan’s code showed (or at least my interpretation of it did) that there were two areas that caused the difference. Once is that he used a slightly different method of computing the Julian date, a key part of the algorithm. His change includes the time of day as part of the calculation. The net result of this change is that solar time generated using his algorithm varies based on the time of day the calculation is run, something that is incorrect. This means that there can be a discrepancy of up to one calendar day. If the user calculated sunrise at 11:59 PM, sunrise would be calculated for the following day even if the user attempted to calculate it for today. In addition, the other calculations do not match the output of the matching NOAA code. I have deprecated the JSuntimeCalculator and in its place added the NOAACalculator that was the result of the direct port of the NOAA code, shoehorned into the Zmanim API Calculator interface. I ran some tests to compare the maximum and minimum discrepancy between the 2 implementations, and calculations for Lakewood, NJ, latitude 40.0828, longitude -74.2094 show a discrepancy of between a minute and 34 seconds to a minute and 37 seconds for sunrise and sunset across an entire year of sunrise and sunset calculations. I also compared the USNO algorithm to the new NOAA implementation and ended up with a maximum deviation of less than 30 seconds, something that had been about 1.5 minutes apart previously. While I do believe that the Julian date calculation is a bug, I do not know that this is a case as far as the rest of the calculation, but it is clear that it does not match the NOAA implementation that is was based on, and I recommend that you download the latest version that has the new NOAACalculator that fixes this issue. In addition to this fix, an additional patch will be released later this week that will address issues with calculations in the arctic circle. Stay posted for the next post.

Technical Information about the Bearing to Yerushalayim Map

Vintage Map with CompassI have been asked a number of questions recently about the directional accuracy of the Bearing to Yerushalayim and Zmanim Map calculations. The rhumb line calculations are very straight forward, and I will not spend much time on it in this article. As a side note, for those interested in the subject, I would like to mention that Rabbi Gavriel Goetz recently published a very comprehensive pamphlet Gevuras Moishe on the subject and it is well worth reading.
My original calculations and implementation of the great circle route (geodesic line) used simple spherical trigonometry to calculate the initial bearing (and distance) based on a sphere using an authalic mean radius of 6371 km. These calculations were very similar to the method used by Rabbi Yehuda Herskowitz’s article in Yeshurun volume III page 586. This model of the earth is more than accurate enough for these calculations (you would need a very accurate “nose alignment” and the total lack of shuckling to even get within a degree of being correct). Google Maps API, Yahoo Maps and Microsoft’s Live Maps for simplicity sake, all use very similar calculations as mentioned in the cfis blog quoting Morten Nielsen

Google Maps / Virtual Earth / Yahoo Maps(?) all use a spherical datum based on WGS84. That is, it has the same center, orientation and scale as WGS84, but has no flattening. The radius of the sphere is the same as the semi-major axis of WGS84 (6378137 meters).

Using a sphere, all the above-mentioned mapping APIs use spherical law of cosines formula for the calculations, that yield an identical result to the more complex Haversine formula.

Curious how accurate these calculations really are, being based on a perfect sphere and not the oblate ellipsoid/spheroid that the earth actually is, I started looking into the subject a little more. The most accurate current geoid model of the earth is the WGS84 model. The geodesist/mathematician Thaddeus Vincenty published the Vincenty formulae for this type of calculation that is said to be accurate to about one-half millimeter, more than adequate for our calculation. Chris Veness implemented this formula in JavaScript and released it under the LGPL, making it very simple to use. I slightly modified his work to allow easier interaction with the Google Maps API. The implementation that I used when I finally published the map on December 30, 2007, uses the Vincenty formula with the WGS84 geoid model (the Vincenty formula can easily be used with different goeiod models). The table below shows a comparison of the results for the initial bearing and distances of the different calculations based an a Lakewood, NJ (latitude: 40.095965°, longitude: -74.22213°) to Har Habayis (latitude: 31.77805°, longitude: 35.235149°) example.

Calculation Method Initial Bearing Distance
6,371 km (authalic mean radius) sphere using spherical trigonometry 53.86555° 9224.67442 km
Google Maps API 6,378.137km (WGS84 equatorial mean radius) sphere using spherical trigonometry N/A 9235.00819 km
Vincenty formula using a WGS84 geoid 53.81786° 9244.61686 km
rhumb line 95.37152° 9891.16074 km

Zmanim Via Instant Messaging Bot

As hinted at in my previous post , there is a new project underway that uses the Zmanim API in a way that I had never really imagined. Using the Smack API, Michael Kopinsky created the ZmanimBot that allows getting zmanim by instant messaging the ZmanimBot, an internet bot. It currently supports the Google Talk IM system, but support for other systems might follow. Please be aware that the system is under development and is not always up. Additional information can be found on the ZmanimBot page.

Update (ג׳ אייר תשע״ג April 13, 2008): The ZmanimBot is now available via AIM