本文共 21807 字,大约阅读时间需要 72 分钟。
Hystrix熔断器
分布式系统面临的问题
复杂分布式体系结构中的应用程序有数10个依赖关系,每个依赖关系在某些时候将不可避免地失败服务雪崩
多个微服务之间调用的时候,假设微服务A调用微服务B和微服务C,微服务B和微服务C又调用其它的微服务,这就是所谓的“扇出” 。如果扇出的链路.上某个微服务的调用响应时间过长或者不可用,对微服务A的调用就会占用越来越多的系统资源,进而引起系统崩溃,所谓的” 雪崩效应”。对于高流量的应用来说,单一的后端依赖可能会导致所有服务器上的所有资源都在几秒钟内饱和。比失败更糟糕的是,这些应用程序还可能导致服务之间的延迟增加,备份队列,线程和其他系统资源紧张,导致整个系统发生更多的级联故障。这些都表示需要对故障和延迟进行隔离和管理,以便单个依赖关系的失败,不能取消整个应用程序或系统。
所以,通常当你发现一个模块下的某个实例失败后,这时候这个模块依然还会接收流量,然后这个有问题的模块还调用了其他的模块,这样就会发生级联故障,或者叫雪崩。
Hystrix是一个用于处理分布式系统的延迟和容错的开源库 ,在分布式系统里,许多依赖不可避免的会调用失败,比如超时、异常等,Hystrix能够保证在一个依赖出问题的情况下, 不会导致整体服务失败,避免级联故障,以提高分布式系统的弹性。
“断路器”本身是一种开关装置, 当某个服务单元发生故障之后,通过断路器的故障监控(类似熔断保险丝),向调用方返回一个符合预期的、可处理的备选响应(FallBack) ,而不是长时间的等待或者抛出调用方无法处理的异常,这样就保证了服务调用方的线程不会被长时间、不必要地占用,从而避免了故障在分布式系统中的蔓延,乃至雪崩。
https://github.com/Netflix/hystrix/wiki
https://github.com/Netflix/hystrix
后果:
服务器忙,请稍后再试,不让客户端等待并立刻返回一个友好提示,fallback
哪些情况会发出降级
类比保险丝达到最大服务访问后,直接拒绝访问,拉闸限电,然后调用服务降级的方法并返回友好提示
就是保险丝:服务熔断->进而服务降级->恢复调用链路秒杀高并发等操作,严禁一窝蜂的过来拥挤,大家排队,一秒钟N个,有序进行
cloud2020 com.atguigu.springcloud 1.0-SNAPSHOT 4.0.0 cloud-provider-hystrix-payment8001 org.springframework.cloud spring-cloud-starter-netflix-hystrix org.springframework.cloud spring-cloud-starter-netflix-eureka-server com.atguigu.springcloud cloud-api-common ${project.version} org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-actuator org.springframework.boot spring-boot-devtools runtime true org.projectlombok lombok true org.springframework.boot spring-boot-starter-test test
server: port: 8001spring: application: name: cloud-provider-hystrix-paymenteureka: client: register-with-eureka: true fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
package com.atguigu.springcloud;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.cloud.netflix.eureka.EnableEurekaClient;/** * @author zzyy * @create 2020/3/6 22:21 **/@SpringBootApplication@EnableEurekaClientpublic class PaymentHystrixMain8001 { public static void main(String[] args) { SpringApplication.run(PaymentHystrixMain8001.class, args); }}
package com.atguigu.springcloud.service;import org.springframework.stereotype.Service;import java.util.concurrent.TimeUnit;/** * @author zzyy * @create 2020/3/6 22:23 **/@Servicepublic class PaymentService { /** * 正常访问 * * @param id * @return */ public String paymentInfo_OK(Integer id) { return "线程池:" + Thread.currentThread().getName() + " paymentInfo_OK,id:" + id + "\t" + "O(∩_∩)O哈哈~"; } /** * 超时访问 * * @param id * @return */ public String paymentInfo_TimeOut(Integer id) { int timeNumber = 3; try { // 暂停3秒钟 TimeUnit.SECONDS.sleep(timeNumber); } catch (InterruptedException e) { e.printStackTrace(); } return "线程池:" + Thread.currentThread().getName() + " paymentInfo_TimeOut,id:" + id + "\t" + "O(∩_∩)O哈哈~ 耗时(秒)" + timeNumber; }}
package com.atguigu.springcloud.controller;import com.atguigu.springcloud.service.PaymentService;import lombok.extern.slf4j.Slf4j;import org.springframework.beans.factory.annotation.Value;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.PathVariable;import org.springframework.web.bind.annotation.RestController;import javax.annotation.Resource;import java.util.concurrent.TimeUnit;/** * @author zzyy * @create 2020/3/6 22:30 **/@RestController@Slf4jpublic class PaymentController { @Resource private PaymentService paymentService; @Value("${server.port}") private String servicePort; /** * 正常访问 * * @param id * @return */ @GetMapping("/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id) { String result = paymentService.paymentInfo_OK(id); log.info("*****result:" + result); return result; } /** * 超时访问 * * @param id * @return */ @GetMapping("/payment/hystrix/timeout/{id}") public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { String result = paymentService.paymentInfo_TimeOut(id); log.info("*****result:" + result); return result; }}
http://localhost:8001/payment/hystrix/ok/31
http://localhost:8001/payment/hystrix/timeout/31
以上述为根基平台,从正确->错误->降级熔断->恢复
上述在非高并发情形下,还能勉强满足 but...
下载地址
https://jmeter.apache.org/download_jmeter.cgihttp://localhost:8001/payment/hystrix/timeout/31
两个都在转圈圈
为什么会被卡死?
tomcat的默认工作线程数被打满了,没有多余的线程来分解压力和处理上面还只是服务提供者8001自己测试,假如此时外部的消费者80也来访问,那消费者只能干等,最终导致消费端80不满意,服务端8001直接被拖死
cloud-consumer-feign-hystrix-order80
cloud-consumer-feign-hystrix-order80
cloud2020 com.atguigu.springcloud 1.0-SNAPSHOT 4.0.0 cloud-consumer-feign-hystrix-order80 org.springframework.cloud spring-cloud-starter-openfeign org.springframework.cloud spring-cloud-starter-netflix-eureka-server com.atguigu.springcloud cloud-api-common ${project.version} org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-actuator org.springframework.boot spring-boot-devtools runtime true org.projectlombok lombok true org.springframework.boot spring-boot-starter-test test
server: port: 80eureka: client: register-with-eureka: false fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
package com.atguigu.springcloud;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.cloud.netflix.eureka.EnableEurekaClient;import org.springframework.cloud.openfeign.EnableFeignClients;/** * @author zzyy * @date 2020/02/18 17:20 **/@SpringBootApplication@EnableEurekaClient@EnableFeignClientspublic class OrderHystrixMain80 { public static void main(String[] args) { SpringApplication.run(OrderHystrixMain80.class, args); }}
package com.atguigu.springcloud.service;import org.springframework.cloud.openfeign.FeignClient;import org.springframework.stereotype.Component;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.PathVariable;/** * @author zzyy * @create 2020/3/6 23:19 **/@Component@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")public interface PaymentHystrixService { /** * 正常访问 * * @param id * @return */ @GetMapping("/payment/hystrix/ok/{id}") String paymentInfo_OK(@PathVariable("id") Integer id); /** * 超时访问 * * @param id * @return */ @GetMapping("/payment/hystrix/timeout/{id}") String paymentInfo_TimeOut(@PathVariable("id") Integer id);}
@RestController@Slf4jpublic class OrderHystrixController { @Resource private PaymentHystrixService paymentHystrixService; @GetMapping("/consumer/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id){ String result = paymentHystrixService.paymentInfo_OK(id); return result; } @GetMapping("/consumer/payment/hystrix/timeout/{id}") public String paymentInfo_Timeout(@PathVariable("id") Integer id){ String result = paymentHystrixService.paymentInfo_Timeout(id); return result; }}
http://localhost/consumer/payment/hystrix/ok/32
开启Jmeter,来20000个并发压死8001
消费者80微服务再去访问的OK服务8001地址
http://localhost/consumer/payment/hystrix/ok/32
消费者80,o(╥﹏╥)o
正因为有上述故障或不佳表现#才有我们的降级/容错/限流等技术诞生
超时不再等待
出错要有兜底
@HystrixCommand
设置自身调用超时时间的峰值,峰值内可以正常运行,#超过了需要有兜底的方法处理,做服务降级fallback
@HystrixCommand(fallbackMethod = "paymentInfo_TimeoutHandler",commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds",value = "3000")})public String paymentInfo_Timeout(Integer id) { int timeNumber = 5; try { TimeUnit.SECONDS.sleep(timeNumber); } catch (Exception e){ e.printStackTrace(); } return "线程池: " + Thread.currentThread().getName() + " paymentInfo_OK,id:" + id + " 耗时(秒):" + timeNumber;}public String paymentInfo_TimeoutHandler(Integer id) { return "/(ToT)/调用支付接口超时或异常、\t" + "\t当前线程池名字" + Thread.currentThread().getName();}
@EnableCircuitBreaker
我们自己配置过的热部署方式对java代码的改动明显,但对@HystrixCommand内属性的修改建议重启微服务
org.springframework.cloud spring-cloud-starter-netflix-hystrix
server: port: 80eureka: client: register-with-eureka: false fetch-registry: true service-url: defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eurekafeign: hystrix: enabled: true
@EnableHystrix
@GetMapping("/consumer/payment/hystrix/timeout/{id}")@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500")})public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { //int age = 10/0; return paymentHystrixService.paymentInfo_TimeOut(id);}public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) { return "我是消费者80,对方支付系统繁忙请10秒种后再试或者自己运行出错请检查自己,o(╥﹏╥)o";}
@DefaultProperties(defaultFallback = ")
1: 1每个方法配置一 个服务降级方法,技术上可以,实际上傻X1: N除了个别重要核心业务有专属,它普通的可以通过@DefaultProperties(defaultFallback= ")统- -跳转到统- 处理结果页面通用的和独享的各自分开,避免了代码膨胀,合理减少了代码量, 0(∩∩)O哈哈~
package com.atguigu.springcloud.controller;import com.atguigu.springcloud.service.PaymentHystrixService;import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;import lombok.extern.slf4j.Slf4j;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.PathVariable;import org.springframework.web.bind.annotation.RestController;import javax.annotation.Resource;/** * @author zzyy * @create 2020/3/6 23:20 **/@RestController@Slf4j@DefaultProperties(defaultFallback = "payment_Global_FallbackMethod")public class OrderHystrixController { @Resource private PaymentHystrixService paymentHystrixService; @GetMapping("/consumer/payment/hystrix/ok/{id}") public String paymentInfo_OK(@PathVariable("id") Integer id) { return paymentHystrixService.paymentInfo_OK(id); } @GetMapping("/consumer/payment/hystrix/timeout/{id}") /*@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = { @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1500") })*/ @HystrixCommand public String paymentInfo_TimeOut(@PathVariable("id") Integer id) { //int age = 10/0; return paymentHystrixService.paymentInfo_TimeOut(id); } public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) { return "我是消费者80,对方支付系统繁忙请10秒种后再试或者自己运行出错请检查自己,o(╥﹏╥)o"; } /** * 全局fallback * * @return */ public String payment_Global_FallbackMethod() { return "Global异常处理信息,请稍后重试.o(╥﹏╥)o"; }}
记得开启这个注解
feign: hystrix: enabled: true
单个eureka先启动7001
PaymentHystrixMain8001启动
正常访问测试:http://localhost/consumer/payment/hystrix/ok/32
故意关闭微服务8001
客户端自己调用提示
此时服务端provider已经down了 ,但是我们做了服务降级处理,#让客户端在服务端不可用时也会获得提示信息而不会挂起耗死服务器一句话就是家里的保险丝
熔断机制概述
熔断机制是应对雪崩效应的一种微服务链路保护机制。 当扇出链路的某个微服务出错不可用或者响应时间太长时,会进行服务的降级,进而熔断该节点微服务的调用,快速返回错误的响应信息。当检测到该节点微服务调用响应正常后,恢复调用链路。在Spring Cloud框架里,熔断机制通过Hystrix实现。Hystrix会监控微服务间调用的状况,
当失败的调用到一定阈值,缺省是5秒内20次调用失败,就会启动熔断机制。熔断机制的注解是@HystrixCommand.大神论文:https://martinfowler.com/bliki/CircuitBreaker.html
// 服务熔断@HystrixCommand(fallbackMethod = "paymentCircuitBreaker_fallback",commandProperties = { @HystrixProperty(name = "circuitBreaker.enabled", value = "true"), //是否开启断路器 @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"), //请求数达到后才计算 @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"), //休眠时间窗 @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "60"), //错误率达到多少跳闸})public String paymentCircuitBreaker(@PathVariable("id") Integer id) { if(id < 0){ throw new RuntimeException("****id 不能为负数"); } String serialNumber = IdUtil.simpleUUID(); return Thread.currentThread().getName() + "\t" + "调用成功,流水号:" + serialNumber;}public String paymentCircuitBreaker_fallback(@PathVariable("id") Integer id){ return "id 不能为负数,请稍后再试, o(╥﹏╥)o id: " + id;}
@GetMapping("/payment/circuit/{id}")public String paymentCircuitBreaker(@PathVariable("id") Integer id) { String result = paymentService.paymentCircuitBreaker(id); log.info("*****result: " + result); return result;}
自测cloud-provider-hystrix-payment8001
正确:http://localhost:8001/payment/circuit/31
错误:http://localhost:8001/payment/circuit/-31一次正确一次错误trytry
重点测试
多次正确,然后慢慢正确,发现刚开始不满足条件,就算是正确的访问也不能进行请求不再调用当前服务,内部设置一般为MTTR(平均故障处理时间),当打开长达导所设时钟则进入半熔断状态
熔断关闭后不会对服务进行熔断
部分请求根据规则调用当前服务,如果请求成功且符合规则则认为当前服务恢复正常,关闭熔断
再有请求调用的时候,将不会调用主逻辑,而是直接调用降级fallback。通过断路器,实现了自动地发现错误并将降级逻辑切换为主逻辑,减少响应延迟的效果。
原来的主逻辑要如何恢复呢?
对于这一-问题,hystrix也为我们实现了 自动恢复功能。当断路器打开,对主逻辑进行熔断之后,hystrix会启动- 个休眠时间窗, 在这个时间窗内, 降级逻辑是临时的成为主逻辑,当休眠时间窗到期,断路器将进入半开状态,释放- -次请求到原来的主逻辑上,如果此次请求正常返回,那么断路器将继续闭合,主逻辑恢复,如果这次请求依然有问题,断路器继续进入打开状态,休眠时间窗重新计时。后面高级篇讲解alibaba的Sentinel说明
https://github.com/Netflix/Hystrix/wiki/How-it-Works
除了隔离依赖服务的调用以外,Hystrix还提供 了准实时的调用监控(Hystrix Dashboard),Hystrix会持续地记录所有通过Hystrix发起的请求的执行信息,以统计报表和图形的形式展示给用户,包括每秒执行多少请求多少成功,多少失败等。Netflix通过hystrix- metrics-event-stream项目实现了对以上指标的监控。Spring Cloud也提供了Hystrix Dashboard的整合,对监控内容转化成可视化界面。
cloud2020 com.atguigu.springcloud 1.0-SNAPSHOT 4.0.0 cloud-consumer-hystrix-dashboard9001 hystrix监控 org.springframework.cloud spring-cloud-starter-netflix-hystrix-dashboard org.springframework.boot spring-boot-starter-actuator org.springframework.boot spring-boot-devtools runtime true org.projectlombok lombok true org.springframework.boot spring-boot-starter-test test
server: port: 9001
package com.atguigu.springcloud;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;/** * @author zzyy * @create 2020/3/7 17:27 **/@SpringBootApplication@EnableHystrixDashboardpublic class HystrixDashboardMain9001 { public static void main(String[] args) { SpringApplication.run(HystrixDashboardMain9001.class); }}
org.springframework.boot spring-boot-starter-actuator
http://localhost:9001/hystrix
package com.atguigu.springcloud;import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.boot.web.servlet.ServletRegistrationBean;import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;import org.springframework.context.annotation.Bean;/** * @author zzyy * @create 2020/3/7 17:27 **/@SpringBootApplication@EnableHystrixDashboardpublic class HystrixDashboardMain9001 { public static void main(String[] args) { SpringApplication.run(HystrixDashboardMain9001.class); } /** * 此配置是为了服务监控而配置,与服务容错本身无观,springCloud 升级之后的坑 * ServletRegistrationBean因为springboot的默认路径不是/hystrix.stream * 只要在自己的项目中配置上下面的servlet即可 * @return */ @Bean public ServletRegistrationBean getServlet(){ HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet(); ServletRegistrationBeanregistrationBean = new ServletRegistrationBean<>(streamServlet); registrationBean.setLoadOnStartup(1); registrationBean.addUrlMappings("/hystrix.stream"); registrationBean.setName("HystrixMetricsStreamServlet"); return registrationBean; }}
填写监控地址
http://localhost:8001/hystrix.streamhttp://localhost:8001/payment/circuit/31
http://localhost:8001/payment/circuit/-31上述测试通过ok先访问正确地址,再访问错误地址,再正确地址,会发现图标断路器都是慢慢放开的.
监控结果,成功
监控结果,失败
实心圆:共有两种含义。它通过颜色的变化代表了实例的健康程度,它的健康度从绿色<黄色<橙色<红色递减。
该实心圆除了颜色的变化之外,它的大小也会根据实例的请求流量发生变化,流量越大该实心圆就越大。所以通过该实心圆的展示,就可以在大量的实例中快速的发现故障实例和高压力实例。曲线:用来记录2分钟内流量的相对变化,可以通过它来观察到流量的上升和下降趋势。
转载地址:http://lntkz.baihongyu.com/